Diffinery

codician/diffinerypublic

Open-source, agent-first GitHub and GitHub Actions alternative that runs on your own Cloudflare account: git hosting, pull requests, code review and drop-in GitHub Actions YAML on Workers, Durable Objects, R2 and Containers.

Headless runs in the jail: model requests through a proxy outside it, only the package registries otherwise (phase 5d) (#172)

* Headless runs in the jail: model requests through a proxy outside it, only the package registries otherwise (phase 5d)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014VDQqT8oL18uLJ9fpsNxW2

* Jail: refuse shared directories that undo it, fail as JAIL_FAILED on any error, cancel upstream on hang-up; CI's agents shard gets bwrap

Security's review of #172:
- DIFFINERY_JAIL_SHARED_FOR_TESTS can no longer turn off the jail's isolation:
  a shared directory that is /, holds a hidden path, or is inside the work root
  or home stops the jail from starting, and a request's env can't set it
- any unexpected error while starting the jail exits JAIL_FAILED, not 1
- the relay closes both sides together, and the model proxy cancels its
  request to Diffinery when the agent hangs up (and after 10 idle minutes)
- the plain-HTTP side sends the Host it allowed

The e2e pools keep their work root and home beside the shared directory.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014VDQqT8oL18uLJ9fpsNxW2

---------

Co-authored-by: Claude <noreply@anthropic.com>
Canberk Demirci authored · GitHub committed signed1 parent 5faec44commit f8fef6f32b21fbbab01596124b32ce8b6aab3861Browse files
12 files changed+553−133
Files (12)
modified.github/workflows/e2e.yml+3−2View file
@@ -43,9 +43,10 @@
4343 - run: pnpm -r --if-present build
4444 - if: matrix.shard == 'rest'
4545 run: pnpm --filter @diffinery/app exec playwright install --with-deps chromium
46- # Terminal tasks run in a bubblewrap jail (containers/runner/src/jail.ts); Ubuntu's AppArmor otherwise keeps
46+ # Agent runs (terminal tasks, and headless runs in the agents shard) are in a bubblewrap jail
47+ # (containers/runner/src/jail.ts); Ubuntu's AppArmor otherwise keeps
4748 # it from creating user namespaces. Diffinery's own runner image already has bwrap and no sudo, running as root.
48- - if: matrix.shard == 'rest'
49+ - if: matrix.shard == 'rest' || matrix.shard == 'agents'
4950 run: |
5051 as_root() { if [ "$(id -u)" = 0 ]; then "$@"; else sudo "$@"; fi; }
5152 if ! command -v bwrap >/dev/null; then
modified.github/workflows/live-test.yml+30−3View file
@@ -507,8 +507,9 @@
507507 curl -sS -X POST "$base/$id/close" -H "Authorization: Bearer $TOKEN" >/dev/null || true
508508 jq -e --arg r "# $REPO" '.exitCode == 0 and (.stdout | contains($r))' >/dev/null <<<"$out" || { echo "::error::the sandbox did not run the command in the repository"; exit 1; }
509509
510- # Owned by the agentic thread: proves a terminal task's jail (containers/runner/src/jail.ts) on Cloudflare.
511- - name: A terminal task's jail (bubblewrap, no network but the allowlist proxy)
510+ # Owned by the agentic thread: proves the agent jails (containers/runner/src/jail.ts) on Cloudflare: a terminal
511+ # task's, and a headless run's (phase 5d), which reaches Diffinery only at its model endpoints.
512+ - name: Agent jails (bubblewrap, no network but the allowlist proxy and the model proxy)
512513 if: steps.token.outputs.skip != 'true'
513514 run: |
514515 set -euo pipefail
@@ -533,7 +534,8 @@
533534 jail=no
534535 for attempt in 1 2 3 4 5; do
535536 open
536- has=$(run '[ -n "${DIFFINERY_JAIL:-}" ] && [ -x "$DIFFINERY_JAIL" ] && echo yes || echo no' 20 | jq -r '.stdout // empty' || true)
537+ # A runner with headless jails asks for the run's model token; an older one ignores --egress.
538+ has=$(run 'mkdir -p ../probe ../probe.out; [ -n "${DIFFINERY_JAIL:-}" ] && [ -x "$DIFFINERY_JAIL" ] && "$DIFFINERY_JAIL" agent --dir ../probe --out ../probe.out --egress model -- true 2>&1 | grep -q DIFFINERY_MODEL_TOKEN && echo yes || echo no' 20 | jq -r '.stdout // empty' || true)
537539 if [ "$has" = yes ]; then jail=yes; break; fi
538540 echo "sandbox $id is on a runner image without the jail; trying another in a minute"
539541 close
@@ -542,6 +544,7 @@
542544 [ "$jail" = yes ] || { echo "::error::the sandbox started on a runner image without the jail (rollout not finished?)"; exit 1; }
543545 inside='echo "direct=$(node -e "require(\"net\").connect(443, \"1.1.1.1\").on(\"connect\", () => (console.log(\"open\"), process.exit())).on(\"error\", (e) => (console.log(e.code), process.exit()))")"
544546 echo "other=$(curl -sS -m 20 -o /dev/null -w "%{http_connect}" https://example.com 2>/dev/null || true)"
547+ echo "sni=$(curl -sS -m 20 -o /dev/null -w "%{http_code}" --connect-to example.com:443:registry.npmjs.org:443 https://example.com 2>/dev/null; true)"
545548 echo "checkout=$(ls -A /workspace/repo | paste -sd,)"
546549 echo "credential=$(cat /workspace/repo/.git/config 2>/dev/null || echo hidden)"
547550 v=$(npm view left-pad version --no-update-notifier --loglevel=error 2>/tmp/npm.err)
@@ -557,6 +560,30 @@
557560 grep -qx 'checkout=.git' <<<"$o" || { echo "::error::the jail sees the person's checkout"; fail=1; }
558561 grep -qx 'credential=hidden' <<<"$o" || { echo "::error::the jail sees the checkout's git config"; fail=1; }
559562 grep -qE '^npm=[0-9]+\.[0-9]+' <<<"$o" || { echo "::error::npm view left-pad didn't work through the proxy"; fail=1; }
563+ grep -qx 'sni=000' <<<"$o" || { echo "::error::the proxy let TLS for example.com through a tunnel to registry.npmjs.org"; fail=1; }
564+
565+ # A headless run's jail: registries only, and Diffinery only at /-/model/ through the model proxy outside,
566+ # which adds the run's token (a stand-in here, so Diffinery answers 401) and drops whatever the agent sends.
567+ host=${DIFFINERY#https://}
568+ headless='echo "model=$(curl -s -o /dev/null -w "%{http_code}" -X POST "$ANTHROPIC_BASE_URL/v1/messages" -H "x-api-key: $ANTHROPIC_API_KEY" -H "content-type: application/json" -d "{}")"
569+ echo "api=$(curl -s -o /dev/null -w "%{http_code}" --path-as-is "$ANTHROPIC_BASE_URL/../../api/v3/user")"
570+ echo "connect=$(curl -sS -m 20 -o /dev/null -w "%{http_connect}" https://'"$host"'/api/v3/user 2>/dev/null || true)"
571+ echo "vendor=$(curl -sS -m 20 -o /dev/null -w "%{http_connect}" https://api.anthropic.com 2>/dev/null || true)"
572+ echo "token=$(env | grep -c dmt_ || true)"
573+ echo "credential=$(cat /workspace/repo/.git/config 2>/dev/null || echo hidden)"
574+ v=$(npm view left-pad version --no-update-notifier --loglevel=error 2>/dev/null)
575+ echo "npm=$(printf "%s\n" "$v" | grep -E "^[0-9]+\.[0-9]+\.[0-9]+$" | tail -1)"'
576+ out=$(run "mkdir -p ../headless-check ../headless-check.out && DIFFINERY_MODEL_TOKEN=dmt_livetest_standin DIFFINERY_MODEL_ORIGIN=$(printf '%q' "$DIFFINERY") ANTHROPIC_BASE_URL=http://127.0.0.1:3129/-/model/anthropic ANTHROPIC_API_KEY=diffinery-jail \"\$DIFFINERY_JAIL\" agent --dir ../headless-check --out ../headless-check.out --egress model -- sh -c $(printf '%q' "$headless")")
577+ echo "$out" | jq -r '"exit \(.exitCode)", .stdout, .stderr'
578+ o=$(jq -r .stdout <<<"$out")
579+ jq -e '.exitCode == 0' >/dev/null <<<"$out" || { echo "::error::the headless jail didn't run (exit $(jq -r .exitCode <<<"$out"))"; fail=1; }
580+ grep -qx 'model=401' <<<"$o" || { echo "::error::the model proxy didn't reach Diffinery's model endpoint with the run's token"; fail=1; }
581+ grep -qx 'api=403' <<<"$o" || { echo "::error::the model proxy forwarded a path outside /-/model/"; fail=1; }
582+ grep -qx 'connect=403' <<<"$o" || { echo "::error::the headless jail could open a tunnel to Diffinery"; fail=1; }
583+ grep -qx 'vendor=403' <<<"$o" || { echo "::error::the headless jail could reach a CLI vendor"; fail=1; }
584+ grep -qx 'token=0' <<<"$o" || { echo "::error::the run's token is inside the headless jail"; fail=1; }
585+ grep -qx 'credential=hidden' <<<"$o" || { echo "::error::the headless jail sees the checkout's git config"; fail=1; }
586+ grep -qE '^npm=[0-9]+\.[0-9]+' <<<"$o" || { echo "::error::npm view left-pad didn't work in the headless jail"; fail=1; }
560587 exit $fail
561588
562589 - name: Pull requests and the merge queue
modifiedapps/diffinery/docs/agents/tasks.md+24−5View file
@@ -36,8 +36,8 @@
3636 | --- | --- |
3737 | Sandbox | Re-checks that you may run the task, then starts a fresh sandbox **as the agent**, on behalf of you, with the base branch (or the task's branch, for later runs) checked out. Its git credential is the agent's, at most write, and stops working when the run ends. |
3838 | Install | Installs the runtime from its vendor's official npm package, at a version Diffinery pins (see [Runtimes](#runtimes)). No model credential exists in the sandbox yet. |
39-| Agent | Runs the runtime headless with the goal, plus the project's instructions and memory when it has them (see [Projects](/docs/agents/projects)). Its process, and only its process, gets a model token for this run (see [How your key is protected](#how-your-key-is-protected)). |
40-| Push | Commits what the agent left uncommitted, as the agent, and pushes `agent/<number>-<slug>`. Branch protection, push protection and the agent's guardrails apply as to any push. |
39+| Agent | Runs the runtime headless with the goal, plus the project's instructions and memory when it has them (see [Projects](/docs/agents/projects)), in a jail with a clone of its own (see [What a run can reach](#what-a-run-can-reach)). Its model requests go through Diffinery's model proxy with a token for this run, which stays outside the jail (see [How your key is protected](#how-your-key-is-protected)). |
40+| Push | Commits what the agent left uncommitted, as the agent, in a jail with no network, and takes the new commits out as a git bundle. The sandbox's checkout then pushes them to `agent/<number>-<slug>`, with no hooks. Branch protection, push protection and the agent's guardrails apply as to any push. |
4141 | Pull request | Opens a pull request from that branch as the agent, with the goal as its `intent`, or updates the one the task already has. |
4242 | Close | Closes the sandbox. The run's git credential and model token stop working. |
4343
@@ -146,7 +146,7 @@
146146
147147 Your key never enters a sandbox:
148148
149-1. When a run's agent starts, Diffinery creates a **model token for that run** (`dmt_...`) and gives it to the runtime's process only, as its API key, with the provider's base URL pointed at Diffinery's model proxy (`https://diffinery.com/-/model/<provider>`).
149+1. When a run's agent starts, Diffinery creates a **model token for that run** (`dmt_...`). The token stays outside the agent's jail, with a small proxy there that forwards the agent's model requests, and only those (`/-/model/...`), to Diffinery's model proxy with the token added. Inside, the runtime's API key is a stand-in (`diffinery-jail`), and the provider's base URL points at that proxy (`http://127.0.0.1:3129/-/model/<provider>`). Whatever credential the agent sends is dropped.
150150 2. The proxy accepts the token only while that run's agent is running. On every request it re-checks that you can still run the task, and that whoever pays still may (see [Whose key pays](/docs/agents/organizations#whose-key-pays)). Then it forwards the request to the provider with the key added, decrypted for that request only.
151151 3. The proxy forwards only the model endpoints the runtimes use, and refuses everything else with `403`, so an agent can't reach the rest of your provider account (stored files, batches, assistants):
152152
@@ -160,12 +160,27 @@
160160
161161 Everything the sandbox returns, including the transcript, masks the model token and the git credential.
162162
163-**What this doesn't protect against.** Anything the agent runs in its sandbox (build scripts, tests, dependencies, code from the repository) can read the model token and the git credential while the run lasts, and repository content can try to trick the agent into misusing them ("prompt injection"). Sandboxes have internet access (only HTTP and HTTPS on instances that set [`EGRESS=web`](/docs/self-hosting/configuration#egress)). So:
163+**What this doesn't protect against.** Anything the agent runs in its jail (build scripts, tests, dependencies, code from the repository) can send model requests through the jail's proxy while the run lasts, on your key, and repository content can try to trick the agent into misusing that ("prompt injection"). It can't read the model token or the git credential, and it reaches only the package registries below. So:
164164
165165 - Use a **dedicated API key with a spending limit** for Diffinery, and revoke it with the provider if you suspect misuse.
166-- Run tasks on repositories whose content you trust.
166+- Run tasks on repositories whose content you trust. A steered agent can still publish what it read to npm or RubyGems with a token planted in the repository, and anything it read can end up in its branch and pull request.
167167 - A transcript can show a credential in another form (for example encoded) that masking can't recognise. Transcripts are visible only to people with write access.
168168
169+### What a run can reach
170+
171+Every headless run (a task's, a plan's, a review's and a coordinator's) runs its agent under [bubblewrap](https://github.com/containers/bubblewrap), inside its sandbox, as a [terminal task](/docs/agents/terminal#what-a-task-can-reach) does:
172+
173+| In the jail | |
174+| --- | --- |
175+| Files | The run's own clone of the repository at its start commit (read and write), and the sandbox checkout's git objects (read only). Not the checkout or its git config (which holds the sandbox's git credential), the home directory, or `/tmp`. The rest of the system is read only. |
176+| Processes | Its own: it can't see or signal anything else in the sandbox. No capabilities. |
177+| Network | None of its own. Model requests go to the model proxy outside the jail, on `127.0.0.1:3129`, which forwards only `/-/model/...` to Diffinery. Everything else goes through a proxy (set as `HTTPS_PROXY`) that connects only to the package registries below, by name, on port 443, and only when the TLS handshake asks for that same host. Diffinery itself, the providers' APIs and any other host are refused. |
178+| Environment | Only what the runtime needs: the model proxy's base URLs, the stand-in API keys and the model a route pins. No sandbox tokens. |
179+
180+The registries: npm and Yarn (`registry.npmjs.org`, `registry.yarnpkg.com`), PyPI (`pypi.org`, `files.pythonhosted.org`), Cargo (`index.crates.io`, `static.crates.io`), RubyGems (`rubygems.org`, `index.rubygems.org`), Maven Central (`repo1.maven.org`, `repo.maven.apache.org`) and apt (`deb.debian.org`, `security.debian.org`, `archive.ubuntu.com`, `security.ubuntu.com`, also over plain HTTP for downloads). Go modules aren't reachable, because the Go module proxy fetches any address it's given: vendor them (`go mod vendor`).
181+
182+A refused connection shows in the run's transcript. If the sandbox can't start the jail, the run fails with "The run's jail couldn't start in its sandbox, so nothing ran", and nothing runs: it never falls back to running without one.
183+
169184 Task sandboxes are closed to the [sandbox API](/docs/agents/sandboxes): no call reaches them, for anyone, including the agent's own tokens and repository admins.
170185
171186 ## Who sees what
@@ -252,4 +267,8 @@
252267 | `Pushing the work ... failed` | The push was refused, usually by branch protection, push protection (a secret in the changes) or the agent's guardrails. The transcript shows git's message. |
253268 | The task stopped with "can no longer run it" | You lost write access to the repository, or can no longer manage the agent. |
254269 | Model calls fail with `401 ... its task run has ended` | The run is over; that token no longer works. Start a new run. |
270+| `diffinery: refused a connection to <host>:443 from the task` in the transcript | The agent or a tool it ran tried a host other than the [package registries](#what-a-run-can-reach). Commit what the work needs (vendored dependencies, for example), or do it outside a task. |
271+| `diffinery: refused a connection to <host>:443 from the task: its TLS ClientHello asks for <other host>` (or `names no single server, or hides it (Encrypted Client Hello)`, or `no TLS ClientHello`) | The tunnel opens only when the TLS handshake names the registry the tool asked the proxy for. A tool that sends a different server name, none, or Encrypted Client Hello, or that doesn't start TLS within 10 seconds, is refused. Point the tool at the registry by its own name. |
272+| `diffinery: refused ... (only Diffinery's model endpoints, /-/model/...)` | Something in the run called Diffinery outside the model endpoints through the jail's model proxy. Runs can't reach the rest of Diffinery. |
273+| `The run's jail couldn't start in its sandbox, so nothing ran` | The sandbox couldn't create the jail's namespaces. The transcript says why. Retry; if it keeps happening, tell your instance's administrator. |
255274 | A custom command gets `403 ... isn't available to agent tasks` | The model proxy forwards only the model endpoints listed under [How your key is protected](#how-your-key-is-protected). |
modifiedapps/diffinery/docs/agents/terminal.md+3−3View file
@@ -83,7 +83,7 @@
8383
8484 The registries are there for downloads. PyPI and crates.io publish through other hosts (`upload.pypi.org`, `crates.io`), so a task can't publish there; npm and RubyGems take uploads on the same hosts they serve downloads from (see the risks below).
8585
86-Anything else is refused, and the refusal shows in the task's transcript. Go modules aren't reachable, because the Go module proxy fetches any address it's given. Vendor them (`go mod vendor`) or run Go tasks in a [sandbox of their own](/docs/agents/tasks). Codex signed in with an API key instead of ChatGPT needs `api.openai.com`, which isn't on the list, so use a ChatGPT sign-in.
86+Anything else is refused, and the refusal shows in the task's transcript. Go modules aren't reachable, because the Go module proxy fetches any address it's given. Vendor them (`go mod vendor`). Codex signed in with an API key instead of ChatGPT needs `api.openai.com`, which isn't on the list, so use a ChatGPT sign-in.
8787
8888 If the sandbox can't start the jail, the task fails and nothing runs.
8989
@@ -93,7 +93,7 @@
9393 - **It can send what it read to an allowed host under someone else's account.** For example, the agent could send your sign-in or the repository's code to an AI provider with an attacker's credentials, or publish it to npm or RubyGems with a token planted in the repository.
9494 - **The task pushes to this repository.** Anything it read can end up in its branch, its commits or its pull request.
9595
96-A task in its own sandbox has none of these risks: it holds only a model token for that run, which Diffinery's model proxy accepts. The notice you accept says this too.
96+A task in its own sandbox runs in the same kind of jail, without the first risk: it has no sign-in, and its model token stays outside the jail (see [What a run can reach](/docs/agents/tasks#what-a-run-can-reach)). The notice you accept says this too.
9797
9898 ## Who can reach it
9999
@@ -137,7 +137,7 @@
137137 | What you see | Cause and fix |
138138 | --- | --- |
139139 | No **Give it a task** box, or `the terminal was opened without agent tasks` | The terminal was opened without **Will run agent tasks**. End it and open another with the box ticked. |
140-| `diffinery: refused a connection to <host>:443 from the task (only the hosts in /docs/agents/terminal#what-a-task-can-reach)` in the transcript | The agent or a tool it ran tried a host outside the [list](#what-a-task-can-reach). Work that needs other hosts belongs in a task with a [sandbox of its own](/docs/agents/tasks), or in the terminal itself, outside a task. |
140+| `diffinery: refused a connection to <host>:443 from the task (only the hosts in /docs/agents/terminal#what-a-task-can-reach)` in the transcript | The agent or a tool it ran tried a host outside the [list](#what-a-task-can-reach). Work that needs other hosts belongs in the terminal itself, outside a task. |
141141 | `diffinery: refused a connection to <host>:443 from the task: its TLS ClientHello asks for <other host>` (or `names no single server, or hides it (Encrypted Client Hello)`, or `no TLS ClientHello`) | The tunnel opens only when the TLS handshake names the host the tool asked the proxy for. A tool that sends a different server name, none, or Encrypted Client Hello, or that doesn't start TLS within 10 seconds, is refused. Point the tool at the allowed host by its own name and turn off Encrypted Client Hello in it. |
142142 | `go: ... dial tcp ...` or another Go module download error | Go modules aren't reachable from a terminal task. Commit them with `go mod vendor`, or run the task in a sandbox of its own. |
143143 | The task fails with "The task's jail couldn't start in this terminal's sandbox, so nothing ran." | bubblewrap couldn't start in the sandbox; the transcript has its error. Nothing ran and nothing was pushed. End the terminal, open a new one and run the task again. |
modifiedapps/diffinery/src/agentic/ProjectDO.ts+49−35View file
@@ -1336,8 +1336,9 @@
13361336 // In a terminal the CLI is usually there already, signed in: keep it.
13371337 const bin = TERMINAL_BIN[t.runtime];
13381338 if (r.terminal_id && bin) install = `command -v ${bin} >/dev/null 2>&1 || { ${install}; }`;
1339- // In a terminal it runs outside the repository: an .npmrc there must not choose what gets installed beside the sign-in.
1340- if (!r.proc) return this.startProc(r, "install", install, {}, {}, INSTALL_TIMEOUT_MS, r.terminal_id ? null : `Installing ${RUNTIMES[t.runtime].name}.`, r.terminal_id ? "/" : undefined);
1339+ // It runs outside the jail, beside the sandbox's credential, and outside the repository: an .npmrc there must not
1340+ // choose what gets installed.
1341+ if (!r.proc) return this.startProc(r, "install", install, {}, {}, INSTALL_TIMEOUT_MS, r.terminal_id ? null : `Installing ${RUNTIMES[t.runtime].name}.`, "/");
13411342 const done = await this.poll(r, "setup");
13421343 if (done === null) return;
13431344 if (done !== 0) return this.endRun(this.run(r.task, r.seq)!, "failed", `Installing ${RUNTIMES[t.runtime].name} failed (exit ${done}).`);
@@ -1351,12 +1352,26 @@
13511352 return this.startProc(r, "agent", `"$DIFFINERY_JAIL" agent --dir "$DIFFINERY_WORKTREE" --out "$DIFFINERY_TASK_OUT" --cli ${t.runtime} -- ${terminalCommand(t.runtime)}`, env, {}, timeoutOf(t) - (Date.now() - Date.parse(r.started_at)), null);
13521353 }
13531354 if (!r.proc) {
1354- // The key itself never enters the sandbox: the runtime gets a token for this run that only the model proxy accepts.
1355+ // Headless (phase 5d): the agent runs in a jail (the runner's jail.ts) in a clone of its own, beside the
1356+ // sandbox's checkout and its git credential, reaching only the package registries and Diffinery's model
1357+ // endpoints. The key itself never enters the sandbox: the run gets a model token that only the model proxy
1358+ // accepts, and even that stays outside the jail, with the jail's own model proxy, which adds it to each
1359+ // request. The agent's API keys are stand-ins.
1360+ await this.prepareClone(r, r.start_sha ?? "", null);
13551361 const providers = JSON.parse(r.providers) as Provider[];
13561362 const token = `dmt_${t.repo_id}_${randomToken(32)}`;
13571363 this.sql.exec("UPDATE runs SET token_hash = ?, checked_at = ? WHERE task = ? AND seq = ?", await sha256(token), now(), r.task, r.seq);
1358- const { env, secret } = agentEnv(this.env.PUBLIC_URL, providers, token, { runtime: t.runtime, route: { via: t.via, model: t.model } });
1359- return this.startProc(r, "agent", RUNTIMES[t.runtime].command(t.command ?? undefined, t.via ?? undefined, t.kind !== "task"), { ...env, DIFFINERY_TASK_PROMPT: r.prompt }, secret, timeoutOf(t) - (Date.now() - Date.parse(r.started_at)), null);
1364+ const { env, secret } = agentEnv(JAIL_MODEL_URL, providers, JAIL_KEY, { runtime: t.runtime, route: { via: t.via, model: t.model } });
1365+ const cmd = t.runtime === "custom" ? 'bash -c "$DIFFINERY_COMMAND"' : RUNTIMES[t.runtime].command(undefined, t.via ?? undefined, t.kind !== "task");
1366+ return this.startProc(
1367+ r,
1368+ "agent",
1369+ `"$DIFFINERY_JAIL" agent --dir "$DIFFINERY_WORKTREE" --out "$DIFFINERY_TASK_OUT" --egress model -- ${cmd}`,
1370+ { ...taskDirs(r), ...env, ...secret, DIFFINERY_TASK_PROMPT: r.prompt, ...(t.runtime === "custom" ? { DIFFINERY_COMMAND: t.command ?? "true" } : {}), DIFFINERY_MODEL_ORIGIN: this.env.PUBLIC_URL.replace(/\/$/, "") },
1371+ { DIFFINERY_MODEL_TOKEN: token },
1372+ timeoutOf(t) - (Date.now() - Date.parse(r.started_at)),
1373+ null,
1374+ );
13601375 }
13611376 if (!r.checked_at || Date.parse(r.checked_at) + RECHECK_MS < Date.now()) {
13621377 await this.checkAccess(t);
@@ -1368,7 +1383,7 @@
13681383 const run = this.run(r.task, r.seq)!;
13691384 // The agent is done: its model token stops working now, before anything else runs.
13701385 this.sql.exec("UPDATE runs SET exit_code = ?, token_hash = NULL WHERE task = ? AND seq = ?", done, r.task, r.seq);
1371- if (r.terminal_id && done === JAIL_FAILED) return this.endRun(this.run(r.task, r.seq)!, "failed", "The task's jail couldn't start in this terminal's sandbox, so nothing ran. The transcript says why.");
1386+ if (done === JAIL_FAILED) return this.endRun(this.run(r.task, r.seq)!, "failed", `The ${r.terminal_id ? "task's jail couldn't start in this terminal's sandbox" : "run's jail couldn't start in its sandbox"}, so nothing ran. The transcript says why.`);
13721387 this.note(r.task, r.seq, `${done === 0 ? `${RUNTIMES[t.runtime].name} finished.` : `${RUNTIMES[t.runtime].name} exited with code ${done}.`}${run.model_calls ? ` ${run.model_calls} model ${run.model_calls === 1 ? "request" : "requests"}, ${formatTokens(run.tokens)} tokens.` : ""}`);
13731388 return this.setStep(run, "finish");
13741389 }
@@ -1378,7 +1393,7 @@
13781393 if (r.kind === "plan") return this.keepPlan(t, r);
13791394 const branch = r.branch ?? t.branch;
13801395 if (!r.proc) {
1381- return this.startProc(r, "finish", r.terminal_id ? TERMINAL_FINISH_SCRIPT : FINISH_SCRIPT, { DIFFINERY_TASK_BRANCH: branch, DIFFINERY_TASK_START: r.start_sha ?? "", DIFFINERY_TASK_MESSAGE: commitMessage(t, r), ...(r.terminal_id ? { ...taskDirs(r), DIFFINERY_TASK_COMMIT: TERMINAL_COMMIT_SCRIPT } : {}) }, {}, 10 * 60 * 1000, `Pushing to ${branch}.`);
1396+ return this.startProc(r, "finish", FINISH_SCRIPT, { DIFFINERY_TASK_BRANCH: branch, DIFFINERY_TASK_START: r.start_sha ?? "", DIFFINERY_TASK_MESSAGE: commitMessage(t, r), ...taskDirs(r), DIFFINERY_TASK_COMMIT: COMMIT_SCRIPT }, {}, 10 * 60 * 1000, `Pushing to ${branch}.`);
13821397 }
13831398 const done = await this.poll(r, "setup");
13841399 if (done === null) return;
@@ -1447,24 +1462,33 @@
14471462 const startSha = branchSha ?? (await stub.getRef(`refs/heads/${t.base}`));
14481463 if (!startSha) throw new HttpError(422, `the branch ${t.base} doesn't exist`);
14491464 this.note(t.number, r.seq, `Checking out ${ref} in ${t.person_login}'s terminal, in a clone of its own.`);
1465+ await this.prepareClone(r, startSha, ref);
1466+ if (this.run(t.number, r.seq)?.step !== "boot") return;
1467+ this.sql.exec("UPDATE runs SET start_sha = ?, step = 'install', proc = NULL, checked_at = ? WHERE task = ? AND seq = ?", startSha, now(), t.number, r.seq);
1468+ }
1469+
1470+ /**
1471+ * A run's own clone (taskDirs), beside the sandbox's checkout and borrowing its objects (`git clone --shared`), at
1472+ * the run's start commit, after fetching `ref` into the checkout (a terminal's checkout is the person's, on any
1473+ * branch). The checkout and its .git, which holds the sandbox's git credential, stay out of the run's jail.
1474+ */
1475+ private async prepareClone(r: RunRow, startSha: string, ref: string | null): Promise<void> {
14501476 // Branch names may hold `$`, backticks, `;` or `|`: everything reaches the script through its environment, quoted.
14511477 const prepared = (await this.sandbox(r, "/exec", {
14521478 cmd: [
14531479 "set -e",
14541480 'g() { git -c core.hooksPath=/dev/null -c core.fsmonitor=false "$@"; }',
1455- 'g fetch -q --no-write-fetch-head origin "refs/heads/$DIFFINERY_TASK_REF"',
1481+ 'if [ -n "$DIFFINERY_TASK_REF" ]; then g fetch -q --no-write-fetch-head origin "refs/heads/$DIFFINERY_TASK_REF"; fi',
14561482 'rm -rf "$DIFFINERY_WORKTREE" "$DIFFINERY_TASK_OUT"',
14571483 'g clone -q --shared --no-checkout --no-tags . "$DIFFINERY_WORKTREE"',
14581484 'g -C "$DIFFINERY_WORKTREE" checkout -q --detach "$DIFFINERY_TASK_START"',
14591485 'g -C "$DIFFINERY_WORKTREE" remote remove origin',
14601486 'mkdir "$DIFFINERY_TASK_OUT"',
14611487 ].join("\n"),
1462- env: { DIFFINERY_TASK_REF: ref, DIFFINERY_TASK_START: startSha, ...taskDirs(r) },
1488+ env: { DIFFINERY_TASK_REF: ref ?? "", DIFFINERY_TASK_START: startSha, ...taskDirs(r) },
14631489 timeoutMs: 5 * 60_000,
14641490 })) as { exitCode: number; stderr: string };
1465- if (prepared.exitCode !== 0) throw new HttpError(422, `checking out ${ref} in the terminal failed: ${prepared.stderr.trim().split("\n").slice(-3).join(" ")}`);
1466- if (this.run(t.number, r.seq)?.step !== "boot") return;
1467- this.sql.exec("UPDATE runs SET start_sha = ?, step = 'install', proc = NULL, checked_at = ? WHERE task = ? AND seq = ?", startSha, now(), t.number, r.seq);
1491+ if (prepared.exitCode !== 0) throw new HttpError(422, `checking out ${ref ?? startSha.slice(0, 7)} for the run failed: ${prepared.stderr.trim().split("\n").slice(-3).join(" ")}`);
14681492 }
14691493
14701494 /** The terminal a run is in must still be open, the task's person's, with the opt-in in force. */
@@ -2048,13 +2072,17 @@
20482072 }
20492073
20502074 /**
2051- * A terminal run's own clone and the directory its finish writes a bundle to, beside the terminal's checkout
2052- * (relative to the sandbox's repository directory, where its commands run).
2075+ * A run's own clone and the directory its finish writes a bundle to, beside the sandbox's checkout (relative to the
2076+ * sandbox's repository directory, where its commands run).
20532077 */
20542078 const taskDirs = (r: RunRow): { DIFFINERY_WORKTREE: string; DIFFINERY_TASK_OUT: string } => ({ DIFFINERY_WORKTREE: `../task-${r.task}-${r.seq}`, DIFFINERY_TASK_OUT: `../task-${r.task}-${r.seq}.out` });
20552079
20562080 /** What the runner's jail exits with when it couldn't start (containers/runner/src/jail.ts). */
20572081 const JAIL_FAILED = 125;
2082+/** Where a headless run's model requests go in its jail, to the jail's own model proxy (jail.ts MODEL_URL). */
2083+const JAIL_MODEL_URL = "http://127.0.0.1:3129";
2084+/** The API key a headless run's agent is given: a stand-in, which the jail's model proxy replaces with the run's token. */
2085+const JAIL_KEY = "diffinery-jail";
20582086
20592087 /** The CLI a terminal run looks for before installing it. */
20602088 const TERMINAL_BIN: Partial<Record<RuntimeId, string>> = { "claude-code": "claude", codex: "codex" };
@@ -2073,34 +2101,20 @@
20732101 }
20742102
20752103 /**
2076- * Commit what the agent left uncommitted, then push to the task's branch if
2077- * anything changed since the run started. The sandbox's own git credential (the
2078- * agent's, through branch protection and push protection) does the push.
2104+ * A run's finish. The agent's clone is the agent's: anything in it (hooks, config, filters) is its to set, so it
2105+ * is only touched in a jail with no network and no credential, which commits what's left and writes a bundle of
2106+ * the new commits. Outside, the sandbox's checkout (which has the credential) takes the bundle's objects as data
2107+ * and pushes them, with no hooks and no fsmonitor: the sandbox's own git credential (the agent's, through branch
2108+ * protection and push protection) does the push.
20792109 */
2080-const FINISH_SCRIPT = [
2110+const COMMIT_SCRIPT = [
20812111 "set -e",
2082- 'git add -A',
2083- 'if ! git diff --cached --quiet; then git commit -q -m "$DIFFINERY_TASK_MESSAGE"; fi',
2084- 'head=$(git rev-parse HEAD)',
2085- 'if [ "$head" = "$DIFFINERY_TASK_START" ]; then echo "diffinery: nothing to push"; exit 0; fi',
2086- 'git push -q origin "HEAD:refs/heads/$DIFFINERY_TASK_BRANCH" 2>&1',
2087- 'echo "diffinery: pushed $head"',
2088-].join("\n");
2089-
2090-/**
2091- * A terminal run's finish. The agent's clone is the agent's: anything in it (hooks, config, filters) is its to
2092- * set, so it is only touched in a jail with no network and no credential, which commits what's left and writes
2093- * a bundle of the new commits. Outside, the terminal's checkout (which has the credential) takes the bundle's
2094- * objects as data and pushes them, with no hooks and no fsmonitor.
2095- */
2096-const TERMINAL_COMMIT_SCRIPT = [
2097- "set -e",
20982112 "git add -A",
20992113 'if ! git diff --cached --quiet; then git commit -q -m "$DIFFINERY_TASK_MESSAGE"; fi',
21002114 'if [ "$(git rev-parse HEAD)" != "$DIFFINERY_TASK_START" ]; then git bundle create -q "$DIFFINERY_TASK_OUT/task.bundle" HEAD "^$DIFFINERY_TASK_START"; fi',
21012115 ].join("\n");
21022116
2103-const TERMINAL_FINISH_SCRIPT = [
2117+const FINISH_SCRIPT = [
21042118 "set -e",
21052119 'rm -rf "$DIFFINERY_TASK_OUT/task.bundle"',
21062120 '"$DIFFINERY_JAIL" finish --dir "$DIFFINERY_WORKTREE" --out "$DIFFINERY_TASK_OUT" -- sh -c "$DIFFINERY_TASK_COMMIT"',
modifiedapps/diffinery/test/e2e/agents-harness.mjs+34−5View file
@@ -15,8 +15,13 @@
1515
1616 export const APP = resolve(dirname(fileURLToPath(import.meta.url)), "../..");
1717 export const AGENT = resolve(APP, "../../containers/runner/dist/agent.mjs");
18-export const work = mkdtempSync(join(tmpdir(), "diffinery-tasks-e2e-"));
19-export const poolRoot = join(work, "pool");
18+// The test's files (scripts, fake CLIs, what commands write) are in `work`, which jails share; the pool's work root
19+// and home are beside it, since a jail refuses to share a directory that holds what it hides.
20+const top = mkdtempSync(join(tmpdir(), "diffinery-tasks-e2e-"));
21+export const work = join(top, "work");
22+mkdirSync(work);
23+export const poolRoot = join(top, "pool");
24+export const poolHome = join(top, "home");
2025 // The terminal test speaks WebSocket with headers (Origin, cookies), which Node's built-in client can't set.
2126 export const { WebSocket: WS } = createRequire(resolve(APP, "../../containers/runner/package.json"))("ws");
2227 export const RIVAL_SESSION = `rival-${randomBytes(12).toString("hex")}`;
@@ -91,6 +96,30 @@
9196 return result;
9297 }
9398
99+/**
100+ * A headless run's model token (phase 5d). The agent in its jail has only a stand-in, so the test reads the token
101+ * where only the runner has it: the environment of the jail's own process outside, found by a path in its command.
102+ * `file` is one the run's command writes $ANTHROPIC_API_KEY to, once it has started.
103+ */
104+export async function jailToken(file) {
105+ await until(`the agent to write ${file}`, async () => existsSync(file));
106+ assert.equal(readFileSync(file, "utf8"), "diffinery-jail", "the agent holds a stand-in, never the run's token");
107+ return until(`the jail of the run writing ${file}`, async () => {
108+ for (const pid of readdirSync("/proc").filter((p) => /^\d+$/.test(p))) {
109+ try {
110+ const argv = readFileSync(`/proc/${pid}/cmdline`, "utf8").split("\0");
111+ if (!argv.includes("--jail") || !argv.some((a) => a.includes(file))) continue;
112+ const env = readFileSync(`/proc/${pid}/environ`, "utf8").split("\0");
113+ const tok = env.find((e) => e.startsWith("DIFFINERY_MODEL_TOKEN="));
114+ if (tok) return tok.slice("DIFFINERY_MODEL_TOKEN=".length);
115+ } catch {
116+ // gone, or not ours
117+ }
118+ }
119+ return null;
120+ });
121+}
122+
94123 /** The sandboxes the pool has started, newest first. */
95124 export const poolSandboxes = () =>
96125 existsSync(poolRoot)
@@ -147,13 +176,13 @@
147176 { mode: 0o755 },
148177 );
149178 // The pool's home is the terminal's: a provider sign-in, and shell history that a task's jail must not show.
150- const poolHome = join(work, "home");
151179 mkdirSync(join(poolHome, ".claude"), { recursive: true });
152180 writeFileSync(join(poolHome, ".claude", ".credentials.json"), '{"fake":"signin"}');
153181 writeFileSync(join(poolHome, ".bash_history"), "HISTORY-SECRET\n");
154182 const poolPort = await freePort();
155183 const poolSecret = randomBytes(16).toString("hex");
156- pool = spawn("node", [AGENT, "--sandbox-pool", "--work-root", poolRoot], { env: { ...process.env, HOME: poolHome, PATH: `${fakeBin}:${process.env.PATH}`, DIFFINERY_SANDBOX_POOL_SECRET: poolSecret, DIFFINERY_SANDBOX_PORT: String(poolPort) }, stdio: ["ignore", "pipe", "pipe"], detached: true });
184+ // Commands share the test's directory with their jails: their scripts are there, and what they write for the test to read.
185+ pool = spawn("node", [AGENT, "--sandbox-pool", "--work-root", poolRoot], { env: { ...process.env, HOME: poolHome, PATH: `${fakeBin}:${process.env.PATH}`, DIFFINERY_SANDBOX_POOL_SECRET: poolSecret, DIFFINERY_SANDBOX_PORT: String(poolPort), DIFFINERY_JAIL_SHARED_FOR_TESTS: work }, stdio: ["ignore", "pipe", "pipe"], detached: true });
157186 pool.stdout.on("data", (d) => (poolLog += d));
158187 pool.stderr.on("data", (d) => (poolLog += d));
159188 await until("sandbox pool", async () => poolLog.includes("listening"), 30_000);
@@ -203,5 +232,5 @@
203232 }
204233 }
205234 upstream?.close();
206- if (process.env.E2E_KEEP !== "1") rmSync(work, { recursive: true, force: true });
235+ if (process.env.E2E_KEEP !== "1") rmSync(top, { recursive: true, force: true, maxRetries: 5, retryDelay: 200 });
207236 });
modifiedapps/diffinery/test/e2e/review.test.mjs+15−12View file
@@ -22,8 +22,11 @@
2222
2323 const APP = resolve(dirname(fileURLToPath(import.meta.url)), "../..");
2424 const AGENT = resolve(APP, "../../containers/runner/dist/agent.mjs");
25-const work = mkdtempSync(join(tmpdir(), "diffinery-review-e2e-"));
26-const poolRoot = join(work, "pool");
25+// Jails share `work` (the stand-in reviewer's scripts and what it writes); the pool's work root is beside it.
26+const top = mkdtempSync(join(tmpdir(), "diffinery-review-e2e-"));
27+const work = join(top, "work");
28+mkdirSync(work);
29+const poolRoot = join(top, "pool");
2730 const ANTHROPIC_KEY = `sk-ant-api03-${randomBytes(30).toString("hex")}`;
2831 let base;
2932 let dev;
@@ -145,7 +148,7 @@
145148
146149 const poolPort = await freePort();
147150 const poolSecret = randomBytes(16).toString("hex");
148- pool = spawn("node", [AGENT, "--sandbox-pool", "--work-root", poolRoot], { env: { ...process.env, DIFFINERY_SANDBOX_POOL_SECRET: poolSecret, DIFFINERY_SANDBOX_PORT: String(poolPort) }, stdio: ["ignore", "pipe", "pipe"], detached: true });
151+ pool = spawn("node", [AGENT, "--sandbox-pool", "--work-root", poolRoot], { env: { ...process.env, DIFFINERY_SANDBOX_POOL_SECRET: poolSecret, DIFFINERY_SANDBOX_PORT: String(poolPort), DIFFINERY_JAIL_SHARED_FOR_TESTS: work }, stdio: ["ignore", "pipe", "pipe"], detached: true });
149152 pool.stdout.on("data", (d) => (poolLog += d));
150153 pool.stderr.on("data", (d) => (poolLog += d));
151154 await until("sandbox pool", async () => poolLog.includes("listening"), 30_000);
@@ -188,7 +191,7 @@
188191 }
189192 }
190193 upstream?.close();
191- if (process.env.E2E_KEEP !== "1") rmSync(work, { recursive: true, force: true });
194+ if (process.env.E2E_KEEP !== "1") rmSync(top, { recursive: true, force: true });
192195 });
193196
194197 /** A stand-in reviewer (and fixer) for the custom runtime: what Claude Code or Codex would print. */
@@ -202,15 +205,13 @@
202205 printf '%s\\n' "$DIFFINERY_TASK_PROMPT" > ${promptsDir}/prompt-$n.txt
203206 case "$DIFFINERY_TASK_PROMPT" in
204207 "You are reviewing"*)
205- # The review's sandbox is read-only: its git credential can't push.
208+ # The review runs in a jail (phase 5d), in a clone with no remote: it can't push.
206209 git -c user.name=probe -c user.email=probe@example.com commit -q --allow-empty -m probe
207210 if git push -q origin HEAD:refs/heads/review-probe 2>/dev/null; then echo "PUSHED"; else echo "push refused"; fi
208- # Nor can its credential write through the API.
209- url=$(git remote get-url origin)
210- root=$(printf %s "$url" | sed -E 's#^(https?://[^/]+)/.*#\\1#')
211- repo=$(printf %s "$url" | sed -E 's#^https?://[^/]+/(.*)\\.git$#\\1#')
212- hdr=$(git config --get-regexp 'http\\..*\\.extraheader' | head -1 | cut -d' ' -f2-)
213- code=$(curl -s -o /dev/null -w '%{http_code}' -X PUT -H "$hdr" -H 'content-type: application/json' "$root/api/v3/repos/$repo/contents/probe.txt" -d '{"message":"probe","content":"cHJvYmU="}')
211+ # The sandbox's checkout, with its git credential, isn't there either.
212+ if git config --get-regexp 'http\\..*\\.extraheader' >/dev/null || [ -e ../repo/.git/config ]; then echo "credential seen"; else echo "no credential"; fi
213+ # Nor can it reach Diffinery's API: its proxy refuses anything but the model endpoints.
214+ code=$(curl -s -o /dev/null -w '%{http_code}' --noproxy '' -X PUT -H 'content-type: application/json' "${base}/api/v3/user" -d '{}')
214215 echo "contents PUT $code"
215216 curl -sS -X POST "$ANTHROPIC_BASE_URL/v1/messages" -H "x-api-key: $ANTHROPIC_API_KEY" -H "content-type: application/json" -d '{"model":"m","messages":[]}'; echo
216217 if printf %s "$DIFFINERY_TASK_PROMPT" | grep -q "still open"; then
@@ -297,7 +298,9 @@
297298 assert.equal(first.task.kind, "review");
298299 assert.match(text, /push refused/, text);
299300 assert.ok(!text.includes("PUSHED"), "a review can't push");
300- assert.match(text, /contents PUT 40[34]/, "nor write through the API");
301+ assert.match(text, /no credential/, "the sandbox's credential is out of the review's reach");
302+ assert.ok(!text.includes("credential seen"));
303+ assert.match(text, /contents PUT 403/, "nor write through the API");
301304 assert.equal((await api("GET", `/repos/${owner}/review-demo/contents/probe.txt`)).status, 404);
302305 assert.equal((await api("GET", `/repos/${owner}/review-demo/branches/review-probe`)).status, 404);
303306 assert.match(text, /fake model says hi/, "the review used the model proxy");
modifiedapps/diffinery/test/e2e/tasks.test.mjs+30−21View file
@@ -38,6 +38,7 @@
3838 poolLog,
3939 token,
4040 modelRequests,
41+ jailToken,
4142 freePort,
4243 api,
4344 ok,
@@ -91,13 +92,21 @@
9192 const bot = (await asPerson("POST", "/diffinery/agents", { login: "task-bot", name: "Task Bot" })).json.token.token;
9293
9394 const tokFile = join(work, "model-token");
94- // The custom runtime: call the model through the proxy, show the key variable (masked), save the token for later, change a file.
95+ // The custom runtime: call the model through the proxy, show the key variable, save it for later, change a file.
96+ // It runs in a jail (phase 5d): what it can see and reach is shown too.
9597 const command = [
9698 `curl -sS -X POST "$ANTHROPIC_BASE_URL/v1/messages" -H "x-api-key: $ANTHROPIC_API_KEY" -H "content-type: application/json" -d '{"model":"m","messages":[]}'`,
9799 "echo",
98100 'echo "key is $ANTHROPIC_API_KEY"',
99101 `printf %s "$ANTHROPIC_API_KEY" > ${tokFile}`,
100102 'echo "goal: $DIFFINERY_TASK_PROMPT" | tail -1',
103+ 'echo "jail: credential=$(cat ../repo/.git/config 2>/dev/null || echo hidden)"',
104+ 'echo "jail: token=$(env | grep -c dmt_)"',
105+ // Only the model endpoints through the jail's model proxy, however the path is spelled; Diffinery itself is out of reach.
106+ `echo "jail: api=$(curl -s -o /dev/null -w '%{http_code}' --path-as-is "$ANTHROPIC_BASE_URL/../../api/v3/user")"`,
107+ // (The test's Diffinery is on 127.0.0.1, which NO_PROXY sends direct; --noproxy '' sends it through the proxy.)
108+ `echo "jail: via-proxy=$(curl -s -o /dev/null -w '%{http_code}' --noproxy '' ${base}/api/v3/user)"`,
109+ `echo "jail: direct=$(curl -s -o /dev/null -w '%{http_code}' --noproxy '*' --max-time 5 ${base}/api/v3/user)"`,
101110 "echo hello > hello.txt",
102111 ].join("; ");
103112 const goal = "Add a hello file";
@@ -137,20 +146,22 @@
137146 assert.equal(modelRequests[0].url, "/v1/messages");
138147 assert.ok(!JSON.stringify(modelRequests[0].headers).includes("dmt_"));
139148 assert.match(text, /fake model says hi/);
140- assert.match(text, /key is \*\*\*/, "the run's token is masked in the transcript");
149+ // The agent had a stand-in for a key: the run's token stayed outside its jail, with the jail's model proxy.
150+ assert.match(text, /key is diffinery-jail/);
151+ assert.equal(readFileSync(tokFile, "utf8"), "diffinery-jail");
141152 assert.match(text, /\[out\] Add a hello file/, "the prompt reaches the runtime");
142153 assert.ok(!text.includes(ANTHROPIC_KEY));
143- const runToken = readFileSync(tokFile, "utf8");
144- assert.match(runToken, /^dmt_/);
145- assert.ok(!text.includes(runToken));
146-
147- // The run is over: its model token no longer works.
148- const late = await fetch(`${base}/-/model/anthropic/v1/messages`, { method: "POST", headers: { "x-api-key": runToken, "content-type": "application/json" }, body: "{}" });
149- assert.equal(late.status, 401);
154+ assert.ok(!text.includes("dmt_"));
155+ // Its jail (phase 5d): no sandbox credential, no token, Diffinery reachable only at its model endpoints, no network of its own.
156+ assert.match(text, /jail: credential=hidden/);
157+ assert.match(text, /jail: token=0/);
158+ assert.match(text, /jail: api=403/);
159+ assert.match(text, /jail: via-proxy=403/);
160+ assert.match(text, /jail: direct=000/);
150161 assert.equal(modelRequests.length, 1);
151- // Nor does the proxy take an account token, or anything outside /v1/.
162+ // The proxy doesn't take an account token, or anything outside /v1/.
152163 assert.equal((await fetch(`${base}/-/model/anthropic/v1/messages`, { method: "POST", headers: { authorization: `Bearer ${token}` } })).status, 401);
153- assert.equal((await fetch(`${base}/-/model/anthropic/admin`, { headers: { "x-api-key": runToken } })).status, 403);
164+ assert.equal((await fetch(`${base}/-/model/anthropic/admin`, { headers: { "x-api-key": `dmt_${"x".repeat(12)}_y` } })).status, 403);
154165
155166 // The pull request is the agent's, from the task's branch, with the work.
156167 const pr = await ok("GET", `/repos/${owner}/tasks-demo/pulls/${done.pull_request.number}`);
@@ -247,8 +258,7 @@
247258 assert.equal(started.status, 201, JSON.stringify(started.json));
248259 const n = started.json.number;
249260 const id = await until("the task's sandbox", async () => poolSandboxes().find((x) => !before_.has(x)) ?? null);
250- await until("the agent to start", async () => existsSync(tokFile));
251- const runToken = readFileSync(tokFile, "utf8");
261+ const runToken = await jailToken(tokFile);
252262
253263 // While it runs, its token works through the proxy.
254264 const live = await fetch(`${base}/-/model/anthropic/v1/messages`, { method: "POST", headers: { "x-api-key": runToken, "content-type": "application/json" }, body: "{}" });
@@ -801,7 +811,10 @@
801811 const P = `/repos/${owner}/tasks-demo/diffinery/project`;
802812 const R = `/repos/${owner}/tasks-demo/diffinery/tasks`;
803813 const src = join(work, "src");
804- // Attempts get the same prompt; each claims a slot (the e2e pool's sandboxes share /tmp) and does different work.
814+ // Attempts get the same prompt; each claims a slot (in the test's work directory, which the jails share) and does
815+ // different work.
816+ const slots = join(work, "best-slots");
817+ mkdirSync(slots, { recursive: true });
805818 writeFileSync(
806819 join(src, "best.sh"),
807820 `p="$DIFFINERY_TASK_PROMPT"
@@ -810,7 +823,7 @@
810823 esac
811824 tag=$(printf '%s\\n' "$p" | grep -o 'BEST-[a-z0-9]*' | head -1)
812825 slot=0
813-for i in 1 2 3 4; do if mkdir "/tmp/$tag-$i" 2>/dev/null; then slot=$i; break; fi; done
826+for i in 1 2 3 4; do if mkdir "${slots}/$tag-$i" 2>/dev/null; then slot=$i; break; fi; done
814827 case $slot in
815828 1) echo one > one.txt; echo SUMMARY-ONE-FILE ;;
816829 2) echo a > a.txt; echo b > b.txt; echo c > c.txt; echo SUMMARY-THREE-FILES ;;
@@ -1005,10 +1018,7 @@
10051018 const cookie = /dfy_csrf=([^;]+)/.exec(res.headers.get("set-cookie") ?? "")?.[1];
10061019 return fetch(`${base}${action}`, { method: "POST", redirect: "manual", headers: { cookie: `dfy_csrf=${cookie}`, "sec-fetch-site": "same-origin" }, body: new URLSearchParams([["_csrf", csrf ?? ""], ...fields]) });
10071020 };
1008- const runToken = async (file) => {
1009- await until(`the agent to write ${file}`, async () => existsSync(join(work, file)));
1010- return readFileSync(join(work, file), "utf8");
1011- };
1021+ const runToken = (file) => jailToken(join(work, file));
10121022 const stopped = (R, n) =>
10131023 until(`task #${n} to stop`, async () => {
10141024 const t = await ok("GET", `${R}/${n}`);
@@ -1221,8 +1231,7 @@
12211231 const started = await asPerson("POST", R, { goal: "Lose access", runtime: "custom", command: `printf %s "$ANTHROPIC_API_KEY" > ${tokFile}; sleep 120`, agent: "task-bot" });
12221232 assert.equal(started.status, 201, JSON.stringify(started.json));
12231233 const n = started.json.number;
1224- await until("the agent to start", async () => existsSync(tokFile));
1225- const runToken = readFileSync(tokFile, "utf8");
1234+ const runToken = await jailToken(tokFile);
12261235 // Disabling the agent ends its access, and with it the task's.
12271236 assert.equal((await api("DELETE", "/diffinery/agents/task-bot")).status, 204);
12281237 const refused = await fetch(`${base}/-/model/anthropic/v1/messages`, { method: "POST", headers: { "x-api-key": runToken, "content-type": "application/json" }, body: "{}" });
modifiedapps/diffinery/test/e2e/terminals.test.mjs+2−1View file
@@ -18,6 +18,7 @@
1818 AGENT,
1919 work,
2020 poolRoot,
21+ poolHome,
2122 WS,
2223 RIVAL_SESSION,
2324 ANTHROPIC_KEY,
@@ -285,7 +286,7 @@
285286 assert.match(t.out(), /No such file/);
286287 assert.match(t.out(), /\{"fake":"signin"\}/, "the terminal's own sign-in is untouched by what the task wrote");
287288 assert.doesNotMatch(t.out(), /attacker/);
288- assert.equal(readFileSync(join(work, "home", ".claude", ".credentials.json"), "utf8"), '{"fake":"signin"}', "byte for byte");
289+ assert.equal(readFileSync(join(poolHome, ".claude", ".credentials.json"), "utf8"), '{"fake":"signin"}', "byte for byte");
289290 assert.match(t.out(), /\n3\r?\n0\r?\n0\r?\n/, "the branch name ran nothing, nor the clone's config outside its jail");
290291 assert.match(t.out(), /\n0\r?\n0\r?\n/, "a clean checkout, and the task's checkout removed");
291292
modifiedcontainers/runner/src/jail.ts+218−41View file
@@ -1,6 +1,7 @@
11 import { accessSync, constants, copyFileSync, existsSync, mkdirSync, mkdtempSync, realpathSync, rmSync, statSync, symlinkSync, writeFileSync } from "node:fs";
22 import { spawn, spawnSync } from "node:child_process";
3-import { createServer, request as httpRequest, type Server } from "node:http";
3+import { createServer, request as httpRequest, type IncomingHttpHeaders, type Server } from "node:http";
4+import { request as httpsRequest } from "node:https";
45 import net from "node:net";
56 import os from "node:os";
67 import path from "node:path";
@@ -24,11 +25,18 @@
2425 * `finish` jails have no way out at all: they commit what the agent left and write a git bundle, which the
2526 * runner, outside, pushes with the sandbox's credential.
2627 *
28+ * Headless runs (phase 5d: tasks, plans, reviews and coordinators in a sandbox of their own) use `--egress model`.
29+ * They have no sign-in and reach no vendor: their CONNECT proxy opens tunnels to the package registries only, and
30+ * their model requests go, as plain HTTP on 127.0.0.1, to a second proxy outside the jail. That one forwards only
31+ * Diffinery's model endpoints (`/-/model/...`) to PUBLIC_URL over TLS, with the run's model token, which it holds
32+ * and the jail never sees: whatever authorization the agent sends is dropped.
33+ *
2734 * agent.mjs --jail --root /workspace agent --dir /workspace/task-1-1 --out /workspace/task-1-1.out --cli claude-code -- claude -p ...
35+ * agent.mjs --jail --root /workspace agent --dir /workspace/task-1-1 --out /workspace/task-1-1.out --egress model -- claude -p ...
2836 */
2937
30-/** Where a terminal task may connect, by name (port 443, or 80 for package mirrors' plain HTTP). */
31-export const JAIL_HOSTS = [
38+/** The CLI vendors' hosts a terminal task reaches with the person's sign-in (port 443). */
39+const CLI_HOSTS = [
3240 // Claude Code: the API, and its sign-in's token refresh.
3341 "api.anthropic.com",
3442 "console.anthropic.com",
@@ -36,7 +44,10 @@
3644 // Codex on a ChatGPT plan: its backend and token refresh.
3745 "chatgpt.com",
3846 "auth.openai.com",
39- // Package registries, download hosts where they have their own (uploads go to other hosts).
47+];
48+
49+/** Package registries, download hosts where they have their own (uploads go to other hosts). */
50+export const REGISTRY_HOSTS = [
4051 "registry.npmjs.org",
4152 "registry.yarnpkg.com",
4253 "pypi.org",
@@ -53,44 +64,66 @@
5364 "security.ubuntu.com",
5465 ];
5566
67+/** Where a terminal task may connect, by name (port 443, or 80 for package mirrors' plain HTTP). */
68+export const JAIL_HOSTS = [...CLI_HOSTS, ...REGISTRY_HOSTS];
69+
70+/** How a jail reaches out: `cli`, a terminal task's (its CLI's vendor and the registries); `model`, a headless run's. */
71+export type JailEgress = "cli" | "model";
72+
5673 /** Hosts the plain-HTTP side of the proxy serves (apt's default sources), for GET and HEAD only. */
5774 const PLAIN_HTTP_HOSTS = new Set(["deb.debian.org", "security.debian.org", "archive.ubuntu.com", "security.ubuntu.com"]);
5875
59-export function jailAllows(host: string, port: number): boolean {
76+export function jailAllows(host: string, port: number, egress: JailEgress = "cli"): boolean {
6077 const h = host.toLowerCase().replace(/\.$/, "");
61- return (port === 443 && JAIL_HOSTS.includes(h)) || (port === 80 && PLAIN_HTTP_HOSTS.has(h));
78+ return (port === 443 && (egress === "model" ? REGISTRY_HOSTS : JAIL_HOSTS).includes(h)) || (port === 80 && PLAIN_HTTP_HOSTS.has(h));
6279 }
6380
6481 /** Exit code when the jail itself can't start: nothing ran. */
6582 export const JAIL_FAILED = 125;
6683
6784 const RELAY_PORT = 3128;
85+/** Where a headless run's model requests go inside its jail: `*_BASE_URL` points here (MODEL_URL). */
86+const MODEL_PORT = 3129;
87+export const MODEL_URL = `http://127.0.0.1:${MODEL_PORT}`;
6888
69-/** Runs inside the jail: listens on 127.0.0.1 for the proxy, then runs the command and exits with it. */
89+/**
90+ * Runs inside the jail: listens on 127.0.0.1, a port per proxy outside (`port=socket`, before the command),
91+ * forwarding each connection to that proxy's Unix socket, then runs the command and exits with it.
92+ */
7093 const RELAY = String.raw`
7194 import net from "node:net";
7295 import os from "node:os";
7396 import { spawn } from "node:child_process";
74-const [sock, port, ...cmd] = process.argv.slice(2);
75-const server = net.createServer((c) => {
76- const u = net.connect(sock);
77- c.pipe(u);
78- u.pipe(c);
79- c.on("error", () => u.destroy());
80- u.on("error", () => c.destroy());
81-});
82-server.on("error", (e) => {
83- process.stderr.write("diffinery: the jail's proxy relay failed: " + e.message + "\n");
84- process.exit(${JAIL_FAILED});
85-});
86-server.listen(Number(port), "127.0.0.1", () => {
87- const child = spawn(cmd[0], cmd.slice(1), { stdio: "inherit" });
88- child.on("error", (e) => {
89- process.stderr.write("diffinery: " + cmd[0] + ": " + e.message + "\n");
90- process.exit(127);
97+const args = process.argv.slice(2);
98+const sep = args.indexOf("--");
99+const pairs = args.slice(0, sep).map((a) => [Number(a.slice(0, a.indexOf("="))), a.slice(a.indexOf("=") + 1)]);
100+const cmd = args.slice(sep + 1);
101+let listening = 0;
102+for (const [port, sock] of pairs) {
103+ const server = net.createServer((c) => {
104+ const u = net.connect(sock);
105+ c.pipe(u);
106+ u.pipe(c);
107+ // Either side hanging up closes the other, so a request the agent gave up on is cancelled outside too.
108+ c.on("close", () => u.destroy());
109+ u.on("close", () => c.destroy());
110+ c.on("error", () => u.destroy());
111+ u.on("error", () => c.destroy());
91112 });
92- child.on("exit", (code, signal) => process.exit(code ?? 128 + (os.constants.signals[signal] ?? 1)));
93-});
113+ server.on("error", (e) => {
114+ process.stderr.write("diffinery: the jail's proxy relay failed: " + e.message + "\n");
115+ process.exit(${JAIL_FAILED});
116+ });
117+ server.listen(port, "127.0.0.1", () => {
118+ if (++listening < pairs.length) return;
119+ const child = spawn(cmd[0], cmd.slice(1), { stdio: "inherit" });
120+ child.on("error", (e) => {
121+ process.stderr.write("diffinery: " + cmd[0] + ": " + e.message + "\n");
122+ process.exit(127);
123+ });
124+ child.on("exit", (code, signal) => process.exit(code ?? 128 + (os.constants.signals[signal] ?? 1)));
125+ });
126+}
94127 `;
95128
96129 export interface JailArgs {
@@ -99,12 +132,13 @@
99132 dir: string;
100133 out: string;
101134 cli?: "claude-code" | "codex";
135+ egress: JailEgress;
102136 cmd: string[];
103137 }
104138
105139 export function parseJailArgs(argv: string[], cwd = process.cwd()): JailArgs {
106140 const sep = argv.indexOf("--");
107- if (sep < 0 || sep === argv.length - 1) throw new Error("usage: --jail --root <dir> agent|finish --dir <dir> --out <dir> [--cli claude-code|codex] -- <command...>");
141+ if (sep < 0 || sep === argv.length - 1) throw new Error("usage: --jail --root <dir> agent|finish --dir <dir> --out <dir> [--cli claude-code|codex | --egress model] -- <command...>");
108142 const opts = argv.slice(0, sep);
109143 const cmd = argv.slice(sep + 1);
110144 const get = (name: string) => {
@@ -116,10 +150,14 @@
116150 const dir = get("--dir");
117151 const out = get("--out");
118152 const cli = get("--cli");
153+ const egress = get("--egress") ?? "cli";
119154 if (!mode || !root || !dir || !out) throw new Error("--jail needs agent or finish, --root, --dir and --out");
120155 if (cli !== undefined && cli !== "claude-code" && cli !== "codex") throw new Error(`unknown --cli ${cli}`);
156+ if (egress !== "cli" && egress !== "model") throw new Error(`unknown --egress ${egress}`);
157+ // A headless run has no sign-in to bring.
158+ if (egress === "model" && (cli || mode !== "agent")) throw new Error("--egress model is for agent jails, without --cli");
121159 const abs = (p: string) => path.resolve(cwd, p);
122- const a: JailArgs = { mode, root: abs(root), dir: abs(dir), out: abs(out), cli, cmd };
160+ const a: JailArgs = { mode, root: abs(root), dir: abs(dir), out: abs(out), cli, egress, cmd };
123161 // The task's own directories: inside the work root, beside the person's checkout and never in it.
124162 const repo = path.join(a.root, "repo");
125163 for (const p of [a.dir, a.out]) {
@@ -156,8 +194,57 @@
156194 };
157195
158196 /** Variables the jail keeps from the sandbox's environment; everything else is cleared. */
159-const KEEP_ENV = ["LANG", "LC_ALL", "GIT_AUTHOR_NAME", "GIT_AUTHOR_EMAIL", "GIT_COMMITTER_NAME", "GIT_COMMITTER_EMAIL", "NODE_EXTRA_CA_CERTS", "REQUESTS_CA_BUNDLE", "SSL_CERT_FILE", "PIP_BREAK_SYSTEM_PACKAGES", "COREPACK_ENABLE_DOWNLOAD_PROMPT", "DIFFINERY_TASK_MESSAGE", "DIFFINERY_TASK_START"];
197+const KEEP_ENV = ["LANG", "LC_ALL", "GIT_AUTHOR_NAME", "GIT_AUTHOR_EMAIL", "GIT_COMMITTER_NAME", "GIT_COMMITTER_EMAIL", "NODE_EXTRA_CA_CERTS", "REQUESTS_CA_BUNDLE", "SSL_CERT_FILE", "PIP_BREAK_SYSTEM_PACKAGES", "COREPACK_ENABLE_DOWNLOAD_PROMPT", "DIFFINERY_TASK_MESSAGE", "DIFFINERY_TASK_START", "DIFFINERY_TASK_PROMPT"];
160198
199+/**
200+ * What a headless run keeps besides (Diffinery's agentEnv): where each provider's model endpoints are, all on
201+ * MODEL_URL, a stand-in for each API key (the real token stays outside), and the model a route pins.
202+ */
203+export const MODEL_ENV = [
204+ "ANTHROPIC_BASE_URL",
205+ "ANTHROPIC_API_KEY",
206+ "ANTHROPIC_AUTH_TOKEN",
207+ "ANTHROPIC_MODEL",
208+ "ANTHROPIC_DEFAULT_OPUS_MODEL",
209+ "ANTHROPIC_DEFAULT_SONNET_MODEL",
210+ "ANTHROPIC_DEFAULT_HAIKU_MODEL",
211+ "ANTHROPIC_SMALL_FAST_MODEL",
212+ "CLAUDE_CODE_SUBAGENT_MODEL",
213+ "OPENAI_BASE_URL",
214+ "OPENAI_API_KEY",
215+ "CODEX_API_KEY",
216+ "OPENROUTER_BASE_URL",
217+ "OPENROUTER_API_KEY",
218+ "DIFFINERY_MODEL",
219+];
220+
221+/** The run's model token and where Diffinery is: read by the jail's model proxy, outside, never kept inside. */
222+export const MODEL_TOKEN_ENV = "DIFFINERY_MODEL_TOKEN";
223+export const MODEL_ORIGIN_ENV = "DIFFINERY_MODEL_ORIGIN";
224+
225+/**
226+ * Directories a test pool shares with every jail, read-write (colon-separated). Only the runner's own environment
227+ * sets it, and only in end-to-end tests, whose commands keep their scripts and results in a temporary directory.
228+ */
229+export const SHARED_ENV = "DIFFINERY_JAIL_SHARED_FOR_TESTS";
230+
231+/**
232+ * A test pool's shared directories, refused (so the jail doesn't start) when one is `/`, holds a hidden path (the
233+ * work root, the home directory, /tmp, /root), or is inside the work root (the checkout and its git credential) or
234+ * the home directory. Shared that way, it would undo the jail's filesystem isolation.
235+ */
236+export function sharedDirs(value: string | undefined, hidden: string[], private_: string[]): string[] {
237+ const dirs = (value ?? "").split(":").filter(Boolean);
238+ for (const d of dirs) {
239+ const real = path.isAbsolute(d) && existsSync(d) ? realpathSync(d) : null;
240+ if (!real) throw new Error(`${SHARED_ENV}: ${d} isn't an existing absolute directory`);
241+ const holds = hidden.some((h) => under(h, real) || under(h, d));
242+ const inside = private_.some((p) => under(real, p) || under(d, p) || (existsSync(p) && under(real, realpathSync(p))));
243+ if (real === "/" || holds || inside) throw new Error(`${SHARED_ENV}: ${d} is, holds or is inside a directory the jail hides`);
244+ }
245+ return dirs;
246+}
247+
161248 /** The bubblewrap command line for a jail (exported for tests). */
162249 export function bwrapArgs(a: JailArgs, o: { home: string; tools: string; env: NodeJS.ProcessEnv; cliPath?: string | null; nodePath: string }): string[] {
163250 // The home directory (shell history, other sign-ins, other checkouts), the work root (the person's checkout and
@@ -166,6 +253,9 @@
166253 const args = ["--unshare-all", "--die-with-parent", "--new-session", "--cap-drop", "ALL", "--ro-bind", "/", "/", "--dev", "/dev", "--proc", "/proc"];
167254 for (const h of hidden) args.push("--tmpfs", h);
168255 const isHidden = (p: string) => hidden.some((h) => under(p, h));
256+ // A test pool's shared directories, with anything hidden inside them hidden again.
257+ const shared = sharedDirs(o.env[SHARED_ENV], hidden, hidden.filter((h) => h !== "/tmp"));
258+ for (const d of shared) args.push("--bind", d, d);
169259 // What the jail does see: the task's clone and its output directory, the checkout's objects read-only.
170260 args.push("--bind", a.dir, a.dir, "--bind", a.out, a.out, "--ro-bind", path.join(a.root, "repo", ".git", "objects"), path.join(a.root, "repo", ".git", "objects"));
171261 // The relay, the proxy's socket and the CLI's link: read-only (connecting to a socket needs no write).
@@ -192,16 +282,76 @@
192282 }
193283 const PATH = [path.join(o.tools, "bin"), ...(o.env.PATH ?? "/usr/local/bin:/usr/bin:/bin").split(":").filter((d) => d && !isHidden(d))].join(":");
194284 args.push("--chdir", a.dir, "--clearenv", "--setenv", "PATH", PATH, "--setenv", "HOME", o.home, "--setenv", "TMPDIR", "/tmp", "--setenv", "DIFFINERY_TASK_OUT", a.out);
195- for (const k of KEEP_ENV) if (o.env[k] !== undefined) args.push("--setenv", k, o.env[k]!);
285+ for (const k of [...KEEP_ENV, ...(a.egress === "model" ? MODEL_ENV : [])]) if (o.env[k] !== undefined) args.push("--setenv", k, o.env[k]!);
196286 if (a.mode === "agent") {
197287 // Claude Code skips its permission prompts when told it's in a sandbox (it runs as root here).
198288 args.push("--setenv", "IS_SANDBOX", "1", "--setenv", "DISABLE_AUTOUPDATER", "1", "--setenv", "CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC", "1");
199289 const proxy = `http://127.0.0.1:${RELAY_PORT}`;
200290 for (const k of ["HTTPS_PROXY", "HTTP_PROXY", "https_proxy", "http_proxy"]) args.push("--setenv", k, proxy);
291+ // Model requests go straight to the relay's MODEL_URL, not through the CONNECT proxy.
292+ if (a.egress === "model") for (const k of ["NO_PROXY", "no_proxy"]) args.push("--setenv", k, "127.0.0.1,localhost");
201293 }
202294 return args;
203295 }
204296
297+const MODEL_IDLE_MS = 10 * 60_000;
298+
299+/** Request headers the model proxy never forwards: hop-by-hop, the agent's own credentials and cookies, proxy headers. */
300+const MODEL_DROP = new Set(["host", "connection", "keep-alive", "proxy-connection", "proxy-authorization", "transfer-encoding", "te", "trailer", "upgrade", "authorization", "x-api-key", "cookie", "forwarded", "x-forwarded-for", "x-forwarded-host", "x-forwarded-proto"]);
301+
302+/**
303+ * The model proxy outside a headless run's jail: plain HTTP on a Unix socket in, Diffinery's model endpoints out.
304+ * It forwards only paths under `/-/model/` (as sent: one that a URL parser would rewrite, with `..` or encoded
305+ * dots, is refused), to `origin`, with the run's token as its only credential.
306+ */
307+export function startModelProxy(listen: string | number, origin: string, token: string, log: (line: string) => void = (l) => process.stderr.write(`${l}\n`)): Promise<Server> {
308+ const base = new URL(origin);
309+ if ((base.protocol !== "https:" && base.protocol !== "http:") || base.username || base.password) throw new Error(`${MODEL_ORIGIN_ENV} must be an http(s) origin`);
310+ const server = createServer((req, res) => {
311+ const raw = req.url ?? "";
312+ let target: URL | null = null;
313+ try {
314+ target = raw.startsWith("/") && !raw.startsWith("//") ? new URL(raw, base.origin) : null;
315+ } catch {
316+ target = null;
317+ }
318+ const asSent = raw.split("?")[0]!;
319+ if (!target || target.origin !== base.origin || target.pathname !== asSent || !target.pathname.startsWith("/-/model/") || /%2e|%2f|%5c|\\/i.test(asSent)) {
320+ log(`diffinery: refused ${req.method} ${raw.slice(0, 200)} from the run (only Diffinery's model endpoints, /-/model/...)`);
321+ res.writeHead(403, { "content-type": "text/plain" }).end("Only Diffinery's model endpoints (/-/model/...) are reachable here.\n");
322+ req.resume();
323+ return;
324+ }
325+ const headers: IncomingHttpHeaders = {};
326+ for (const [k, v] of Object.entries(req.headers)) if (!MODEL_DROP.has(k) && v !== undefined) headers[k] = v;
327+ headers.authorization = `Bearer ${token}`;
328+ const send = base.protocol === "https:" ? httpsRequest : httpRequest;
329+ const up = send(target, { method: req.method, headers }, (r) => {
330+ const out = { ...r.headers };
331+ delete out["set-cookie"];
332+ res.writeHead(r.statusCode ?? 502, out);
333+ r.pipe(res);
334+ });
335+ // The agent hung up: cancel the request to Diffinery, which stops the provider's answer (and what it costs).
336+ res.on("close", () => {
337+ if (!res.writableFinished) up.destroy();
338+ });
339+ // Nor does an answer that has stalled for 10 minutes hold on (Diffinery frees its slot after 15).
340+ up.setTimeout(MODEL_IDLE_MS, () => up.destroy(new Error("no data for 10 minutes")));
341+ up.on("error", (e) => {
342+ if (res.destroyed) return;
343+ log(`diffinery: the model proxy couldn't reach ${base.host}: ${e.message}`);
344+ if (!res.headersSent) res.writeHead(502, { "content-type": "text/plain" }).end("Diffinery's model proxy couldn't be reached.\n");
345+ else res.destroy();
346+ });
347+ req.pipe(up);
348+ });
349+ return new Promise((resolve, reject) => {
350+ server.once("error", reject);
351+ server.listen(listen, () => resolve(server));
352+ });
353+}
354+
205355 const CLIENT_HELLO_TIMEOUT_MS = 10_000;
206356 /** A ClientHello spread over more than this is refused. */
207357 const MAX_CLIENT_HELLO_BYTES = 64 * 1024;
@@ -281,7 +431,10 @@
281431 listen: string | number,
282432 log: (line: string) => void = (l) => process.stderr.write(`${l}\n`),
283433 connectUpstream: (port: number, host: string, onConnect: () => void) => net.Socket = (port, host, onConnect) => net.connect(port, host, onConnect),
434+ egress: JailEgress = "cli",
284435 ): Promise<Server> {
436+ const hostsDoc = egress === "model" ? "/docs/agents/tasks#what-a-run-can-reach" : "/docs/agents/terminal#what-a-task-can-reach";
437+ const who = egress === "model" ? "an agent run" : "a terminal task";
285438 const server = createServer((req, res) => {
286439 let url: URL;
287440 try {
@@ -291,14 +444,16 @@
291444 return;
292445 }
293446 const port = Number(url.port || 80);
294- if (url.protocol !== "http:" || !jailAllows(url.hostname, port) || (req.method !== "GET" && req.method !== "HEAD")) {
295- log(`diffinery: refused ${req.method} ${url.protocol}//${url.host} from the task (only the hosts in /docs/agents/terminal#what-a-task-can-reach)`);
296- res.writeHead(403, { "content-type": "text/plain" }).end(`${url.host} isn't reachable from a terminal task.\n`);
447+ if (url.protocol !== "http:" || !jailAllows(url.hostname, port, egress) || (req.method !== "GET" && req.method !== "HEAD")) {
448+ log(`diffinery: refused ${req.method} ${url.protocol}//${url.host} from the task (only the hosts in ${hostsDoc})`);
449+ res.writeHead(403, { "content-type": "text/plain" }).end(`${url.host} isn't reachable from ${who}.\n`);
297450 return;
298451 }
299452 const headers = { ...req.headers };
300453 delete headers["proxy-connection"];
301454 delete headers["proxy-authorization"];
455+ // The host the proxy allowed, never another one the tool names in Host.
456+ headers.host = url.host;
302457 const up = httpRequest({ host: url.hostname, port, path: `${url.pathname}${url.search}`, method: req.method, headers }, (r) => {
303458 res.writeHead(r.statusCode ?? 502, r.headers);
304459 r.pipe(res);
@@ -311,9 +466,9 @@
311466 const host = m?.[1] ?? "";
312467 const port = Number(m?.[2] ?? 0);
313468 socket.on("error", () => socket.destroy());
314- if (!m || port !== 443 || !jailAllows(host, port) || net.isIP(host)) {
315- log(`diffinery: refused a connection to ${req.url} from the task (only the hosts in /docs/agents/terminal#what-a-task-can-reach)`);
316- socket.end("HTTP/1.1 403 Forbidden\r\ncontent-type: text/plain\r\n\r\nnot reachable from a terminal task\n");
469+ if (!m || port !== 443 || !jailAllows(host, port, egress) || net.isIP(host)) {
470+ log(`diffinery: refused a connection to ${req.url} from the task (only the hosts in ${hostsDoc})`);
471+ socket.end(`HTTP/1.1 403 Forbidden\r\ncontent-type: text/plain\r\n\r\nnot reachable from ${who}\n`);
317472 return;
318473 }
319474 // The tunnel opens only once the client's ClientHello asks for this same host: CONNECT to an allowed
@@ -374,10 +529,24 @@
374529 // exist is never mistaken for an agent that failed.
375530 const probe = spawnSync(bwrap, ["--unshare-all", "--die-with-parent", "--cap-drop", "ALL", "--ro-bind", "/", "/", "--dev", "/dev", "--proc", "/proc", "--tmpfs", "/tmp", "--", "true"], { encoding: "utf8" });
376531 if (probe.status !== 0) return fail(`bubblewrap can't create its namespaces here: ${(probe.stderr || probe.error?.message || `exit ${probe.status}`).trim()}`);
532+ // A headless run's model token stays out here, with its model proxy: nothing kept in the jail may carry it.
533+ const token = process.env[MODEL_TOKEN_ENV] ?? "";
534+ const origin = process.env[MODEL_ORIGIN_ENV] ?? "";
535+ if (a.egress === "model") {
536+ if (!token || !origin) return fail(`--egress model needs ${MODEL_TOKEN_ENV} and ${MODEL_ORIGIN_ENV}`);
537+ if (!/^https?:\/\/[^/@]+$/.test(origin.replace(/\/$/, ""))) return fail(`${MODEL_ORIGIN_ENV} must be an http(s) origin`);
538+ const leak = [...KEEP_ENV, ...MODEL_ENV].find((k) => process.env[k]?.includes(token));
539+ if (leak) return fail(`${leak} holds the run's model token, which must stay outside the jail`);
540+ }
377541 const home = os.homedir();
378- const tools = mkdtempSync(path.join(a.root, ".jail-"));
379- let server: Server | null = null;
542+ let tools: string;
380543 try {
544+ tools = mkdtempSync(path.join(a.root, ".jail-"));
545+ } catch (e) {
546+ return fail((e as Error).message);
547+ }
548+ const servers: Server[] = [];
549+ try {
381550 // The CLI, found as the terminal finds it, linked into the jail's PATH.
382551 const cliPath = which(a.cmd[0]!, process.env.PATH ?? "");
383552 const real = cliPath ? realpathSync(cliPath) : null;
@@ -389,9 +558,15 @@
389558 let cmd = a.cmd;
390559 if (a.mode === "agent") {
391560 const sock = path.join(tools, "proxy.sock");
392- server = await startJailProxy(sock);
561+ servers.push(await startJailProxy(sock, undefined, undefined, a.egress));
562+ const relays = [`${RELAY_PORT}=${sock}`];
563+ if (a.egress === "model") {
564+ const modelSock = path.join(tools, "model.sock");
565+ servers.push(await startModelProxy(modelSock, origin, token));
566+ relays.push(`${MODEL_PORT}=${modelSock}`);
567+ }
393568 writeFileSync(path.join(tools, "relay.mjs"), RELAY);
394- cmd = [process.execPath, path.join(tools, "relay.mjs"), sock, String(RELAY_PORT), ...a.cmd];
569+ cmd = [process.execPath, path.join(tools, "relay.mjs"), ...relays, "--", ...a.cmd];
395570 }
396571 const args = [...bwrapArgs(a, { home, tools, env: process.env, cliPath: real, nodePath: process.execPath }), "--", ...cmd];
397572 const code = await new Promise<number>((resolve) => {
@@ -403,8 +578,10 @@
403578 child.on("exit", (c, sig) => resolve(c ?? 128 + (os.constants.signals[sig as NodeJS.Signals] ?? 1)));
404579 });
405580 return code;
581+ } catch (e) {
582+ return fail(e instanceof Error ? e.message : String(e));
406583 } finally {
407- server?.close();
584+ for (const s of servers) s.close();
408585 rmSync(tools, { recursive: true, force: true });
409586 }
410587 }
modifiedcontainers/runner/src/sandbox.ts+4−2View file
@@ -7,6 +7,7 @@
77 import { WebSocketServer } from "ws";
88 import { basicAuthHeader } from "./builtins/checkout.js";
99 import { Masker } from "./mask.js";
10+import { SHARED_ENV } from "./jail.js";
1011 import { ProcTable } from "./procs.js";
1112 import { Terminal } from "./terminal.js";
1213 import { exec } from "./util.js";
@@ -148,7 +149,8 @@
148149 const strings = (v: unknown, field: string): Record<string, string> => {
149150 if (v === undefined || v === null) return {};
150151 if (typeof v !== "object" || Array.isArray(v)) throw new HttpErr(400, `\`${field}\` must be an object of strings`);
151- return Object.fromEntries(Object.entries(v).map(([k, x]) => [k, String(x)]));
152+ // Only the pool's own environment shares directories with jails (end-to-end tests), never a request.
153+ return Object.fromEntries(Object.entries(v).filter(([k]) => k !== SHARED_ENV).map(([k, x]) => [k, String(x)]));
152154 };
153155
154156 // The web terminal's shell: interactive, so not CI, and without the sandbox's own credentials.
@@ -183,7 +185,7 @@
183185 const cwd = body.cwd ? resolvePath(body.cwd) : repoDir;
184186 if (!existsSync(cwd)) throw new HttpErr(400, `cwd does not exist: ${cwd}`);
185187 const env = baseEnv();
186- if (body.env && typeof body.env === "object") for (const [k, v] of Object.entries(body.env)) env[k] = String(v);
188+ if (body.env && typeof body.env === "object") for (const [k, v] of Object.entries(body.env)) if (k !== SHARED_ENV) env[k] = String(v);
187189 const timeoutMs = typeof body.timeoutMs === "number" && body.timeoutMs > 0 ? body.timeoutMs : 10 * 60_000;
188190 const started = Date.now();
189191 const r = await exec("bash", ["-c", cmd], { cwd, env, timeoutMs, capture: true, maxCapture: maxOut + 1 });
modifiedcontainers/runner/test/jail.test.ts+141−3View file
@@ -1,15 +1,15 @@
11 import { mkdirSync, writeFileSync } from "node:fs";
2-import { request } from "node:http";
2+import { createServer, request, type Server } from "node:http";
33 import net from "node:net";
44 import path from "node:path";
55 import { afterAll, beforeAll, describe, expect, it } from "vitest";
6-import { bwrapArgs, JAIL_HOSTS, jailAllows, parseJailArgs, startJailProxy } from "../src/jail.js";
6+import { bwrapArgs, JAIL_HOSTS, jailAllows, MODEL_URL, parseJailArgs, REGISTRY_HOSTS, SHARED_ENV, startJailProxy, startModelProxy } from "../src/jail.js";
77 import { tmp } from "./helpers.js";
88
99 describe("a terminal task's jail", () => {
1010 it("takes only task directories beside the checkout", () => {
1111 const a = parseJailArgs(["--root", "/w", "agent", "--dir", "../task-1-1", "--out", "../task-1-1.out", "--cli", "claude-code", "--", "claude", "-p", "x"], "/w/repo");
12- expect(a).toEqual({ mode: "agent", root: "/w", dir: "/w/task-1-1", out: "/w/task-1-1.out", cli: "claude-code", cmd: ["claude", "-p", "x"] });
12+ expect(a).toEqual({ mode: "agent", root: "/w", dir: "/w/task-1-1", out: "/w/task-1-1.out", cli: "claude-code", egress: "cli", cmd: ["claude", "-p", "x"] });
1313 const bad = (dir: string, out = "../o") => () => parseJailArgs(["--root", "/w", "agent", "--dir", dir, "--out", out, "--", "true"], "/w/repo");
1414 expect(bad(".")).toThrow(/isn't a task directory/); // the person's checkout
1515 expect(bad("../repo")).toThrow(/isn't a task directory/);
@@ -118,3 +118,141 @@
118118 });
119119 });
120120 });
121+
122+describe("a headless run's jail (phase 5d)", () => {
123+ const args = (...extra: string[]) => ["--root", "/w", "agent", "--dir", "../task-2-1", "--out", "../task-2-1.out", ...extra, "--", "claude", "-p", "x"];
124+ it("takes --egress model, for agent jails without a sign-in", () => {
125+ expect(parseJailArgs(args("--egress", "model"), "/w/repo").egress).toBe("model");
126+ expect(() => parseJailArgs(args("--egress", "model", "--cli", "claude-code"), "/w/repo")).toThrow(/without --cli/);
127+ expect(() => parseJailArgs(["--root", "/w", "finish", "--dir", "../a", "--out", "../b", "--egress", "model", "--", "true"], "/w/repo")).toThrow(/for agent jails/);
128+ expect(() => parseJailArgs(args("--egress", "web"), "/w/repo")).toThrow(/unknown --egress/);
129+ });
130+
131+ it("reaches the package registries only: no CLI vendor, and never Diffinery's host", () => {
132+ for (const h of ["api.anthropic.com", "console.anthropic.com", "chatgpt.com", "auth.openai.com", "diffinery.example"]) expect(jailAllows(h, 443, "model"), h).toBe(false);
133+ for (const h of REGISTRY_HOSTS) expect(jailAllows(h, 443, "model"), h).toBe(true);
134+ expect(jailAllows("archive.ubuntu.com", 80, "model")).toBe(true);
135+ });
136+
137+ it("keeps the model variables, never the run's token, and sends model requests around the CONNECT proxy", () => {
138+ const a = parseJailArgs(args("--egress", "model"), "/w/repo");
139+ const tools = tmp("jail-tools-");
140+ const env = { PATH: "/usr/bin", ANTHROPIC_BASE_URL: `${MODEL_URL}/-/model/anthropic`, ANTHROPIC_API_KEY: "diffinery-jail", DIFFINERY_MODEL_TOKEN: "dmt_repo_secret", DIFFINERY_MODEL_ORIGIN: "https://d.example", OPENAI_API_KEY: "diffinery-jail" };
141+ const s = bwrapArgs(a, { home: "/home/p", tools, env, nodePath: "/usr/bin/node" }).join(" ");
142+ expect(s).toContain(`--setenv ANTHROPIC_BASE_URL ${MODEL_URL}/-/model/anthropic`);
143+ expect(s).toContain("--setenv ANTHROPIC_API_KEY diffinery-jail");
144+ expect(s).toContain("--setenv OPENAI_API_KEY diffinery-jail");
145+ expect(s).not.toContain("dmt_repo_secret");
146+ expect(s).not.toContain("d.example");
147+ expect(s).toContain("--setenv NO_PROXY 127.0.0.1,localhost");
148+ // A terminal task's jail keeps none of them.
149+ const t = bwrapArgs({ ...a, egress: "cli", cli: "claude-code" }, { home: "/home/p", tools, env, nodePath: "/usr/bin/node" }).join(" ");
150+ expect(t).not.toContain("ANTHROPIC_BASE_URL");
151+ expect(t).not.toContain("NO_PROXY");
152+ });
153+
154+ it("shares a test pool's directories, and refuses one that would undo what the jail hides", () => {
155+ const e2e = tmp("jail-e2e-");
156+ mkdirSync(path.join(e2e, "pool", "s1", "repo"), { recursive: true });
157+ mkdirSync(path.join(e2e, "work"));
158+ const root = path.join(e2e, "pool", "s1");
159+ const home = path.join(e2e, "home");
160+ mkdirSync(path.join(home, "sub"), { recursive: true });
161+ const work = path.join(e2e, "work");
162+ const a = parseJailArgs(["--root", root, "agent", "--dir", "../t", "--out", "../t.out", "--egress", "model", "--", "true"], path.join(root, "repo"));
163+ const args = (shared?: string) => bwrapArgs(a, { home, tools: tmp("jail-tools-"), env: shared === undefined ? {} : { [SHARED_ENV]: shared }, nodePath: "/usr/bin/node" }).join(" ");
164+ const s = args(work);
165+ expect(s.indexOf(`--bind ${work} ${work} `)).toBeGreaterThan(s.indexOf("--tmpfs /tmp "));
166+ expect(args()).not.toContain(`--bind ${work} `);
167+ // `/`, the work root (with the checkout and its git credential), the home directory, /tmp, anything holding them
168+ // or inside the work root or home: shared, they would undo the jail's isolation, so the jail doesn't start.
169+ for (const bad of ["/", e2e, path.join(e2e, "pool"), root, path.join(root, "repo"), home, path.join(home, "sub"), "/tmp", `${work}:/`]) expect(() => args(bad), bad).toThrow(/is, holds or is inside a directory the jail hides/);
170+ for (const bad of ["relative", "/nonexistent"]) expect(() => args(bad), bad).toThrow(/isn't an existing absolute directory/);
171+ });
172+
173+ describe("its model proxy", () => {
174+ let sock: string;
175+ let upstream: Server;
176+ let close: () => void;
177+ const seen: Array<{ method: string; url: string; headers: Record<string, unknown>; body: string }> = [];
178+ const logs: string[] = [];
179+ let slowOpen = 0;
180+ let slowClosed = 0;
181+ const until = async (ok: () => boolean) => {
182+ for (let i = 0; i < 100 && !ok(); i++) await new Promise((r) => setTimeout(r, 20));
183+ expect(ok()).toBe(true);
184+ };
185+ beforeAll(async () => {
186+ upstream = createServer((req, res) => {
187+ // A slow answer: it streams until whoever asked goes away.
188+ if (req.headers["content-length"] === "4") {
189+ slowOpen++;
190+ res.writeHead(200, { "content-type": "text/event-stream" });
191+ res.write("data: thinking\n\n");
192+ res.on("close", () => slowClosed++);
193+ return;
194+ }
195+ let body = "";
196+ req.on("data", (d) => (body += d));
197+ req.on("end", () => {
198+ seen.push({ method: req.method!, url: req.url!, headers: req.headers, body });
199+ res.writeHead(200, { "content-type": "text/event-stream", "set-cookie": "s=1" });
200+ res.write("data: one\n\n");
201+ setTimeout(() => res.end("data: two\n\n"), 20);
202+ });
203+ });
204+ await new Promise<void>((r) => upstream.listen(0, "127.0.0.1", r));
205+ const origin = `http://127.0.0.1:${(upstream.address() as net.AddressInfo).port}`;
206+ sock = path.join(tmp("jail-model-"), "m.sock");
207+ const server = await startModelProxy(sock, origin, "dmt_repo_runtoken", (l) => logs.push(l));
208+ close = () => {
209+ server.close();
210+ upstream.close();
211+ };
212+ });
213+ afterAll(() => close());
214+
215+ const send = (method: string, url: string, headers: Record<string, string> = {}, body?: string) =>
216+ new Promise<{ status: number; body: string; headers: Record<string, unknown> }>((resolve) => {
217+ const r = request({ socketPath: sock, method, path: url, headers: { host: "127.0.0.1:3129", ...headers } }, (res) => {
218+ let b = "";
219+ res.on("data", (d) => (b += d));
220+ res.on("end", () => resolve({ status: res.statusCode ?? 0, body: b, headers: res.headers }));
221+ });
222+ r.on("error", () => resolve({ status: 0, body: "", headers: {} }));
223+ r.end(body);
224+ });
225+
226+ it("forwards model requests with the run's token in place of whatever the agent sent, and streams the answer", async () => {
227+ const r = await send("POST", "/-/model/anthropic/v1/messages?beta=true", { "x-api-key": "diffinery-jail", authorization: "Bearer planted-diffinery-token", cookie: "dfy_session=x", "content-type": "application/json", "anthropic-version": "2023-06-01" }, '{"max_tokens":5}');
228+ expect(r.status).toBe(200);
229+ expect(r.body).toBe("data: one\n\ndata: two\n\n");
230+ expect(r.headers["set-cookie"]).toBeUndefined();
231+ const got = seen.at(-1)!;
232+ expect(got.url).toBe("/-/model/anthropic/v1/messages?beta=true");
233+ expect(got.body).toBe('{"max_tokens":5}');
234+ expect(got.headers.authorization).toBe("Bearer dmt_repo_runtoken");
235+ expect(got.headers["x-api-key"]).toBeUndefined();
236+ expect(got.headers.cookie).toBeUndefined();
237+ expect(got.headers["anthropic-version"]).toBe("2023-06-01");
238+ });
239+
240+ it("cancels the request to Diffinery when the agent hangs up", async () => {
241+ const r = request({ socketPath: sock, method: "POST", path: "/-/model/anthropic/v1/messages", headers: { host: "127.0.0.1:3129" } });
242+ r.on("error", () => {});
243+ r.end("slow");
244+ await until(() => slowOpen === 1);
245+ r.destroy();
246+ await until(() => slowClosed === 1);
247+ });
248+
249+ it("refuses everything else on Diffinery, however the path is spelled", async () => {
250+ const before = seen.length;
251+ for (const url of ["/api/v3/user", "/owner/repo.git/info/refs?service=git-receive-pack", "/-/model/../api/v3/user", "/-/model/%2e%2e/api/v3/user", "/-/model/anthropic/%2E%2E/%2E%2E/api", "/-/model%2fanthropic", "/-/model/a\\..\\..\\api", "//evil.example/-/model/x", "http://evil.example/-/model/anthropic/v1/messages", "/-/modelx/anthropic"]) {
252+ expect((await send("GET", url)).status, url).toBe(403);
253+ }
254+ expect(seen.length).toBe(before);
255+ expect(logs.some((l) => l.includes("refused GET /api/v3/user"))).toBe(true);
256+ });
257+ });
258+});