Diffinery

codician/diffinerypublic

Open-source, agent-first GitHub and GitHub Actions alternative that runs on your own Cloudflare account: git hosting, pull requests, code review and drop-in GitHub Actions YAML on Workers, Durable Objects, R2 and Containers.

Usage and cost per task, a 30-day usage view, and a per-person model request limit (phase 5a) (#158)

* Usage and cost per task, a 30-day usage view, and a per-person model request limit (phase 5a)

Each run's tokens come from the model proxy, or for a terminal on a subscription from what the CLI reported
(Claude Code's result, Codex's turns); cost is only Claude Code's own figure. The task page, REST and MCP show
it per run and in all; Settings → Model access and GET /diffinery/model-usage show 30 days from a D1 ledger
(migration 0030). The proxy holds each paying person to 600 model requests a minute (RL_MODEL).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014VDQqT8oL18uLJ9fpsNxW2

* Usage report names each repository as it was when its runs ended; rate limit docs say it counts per location

Security (5a): joining today's owner and name could show a private repository's new name to someone who lost
access before it was renamed or transferred. The ledger row now keeps owner/name from when the run ended.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014VDQqT8oL18uLJ9fpsNxW2

---------

Co-authored-by: Claude <noreply@anthropic.com>
Canberk Demirci authored · GitHub committed signed1 parent f60a1c3commit 9945254e8d05b874fef12b10e1fcada0ba4cf946Browse files
23 files changed+580−18
Files (23)
modifiedapps/diffinery/docs/agents/tasks.md+17−3View file
@@ -128,8 +128,20 @@
128128 - Only you, signed in in a browser, can save, replace or delete your keys. No token, agent, job or sandbox can.
129129 - Saving a key sends you a security email ("Your Anthropic API key was added to Diffinery"), which you can't turn off.
130130 - Keys belong to people. Agents never have model keys of their own, and organization keys aren't available yet.
131-- Subscriptions (Claude Pro or Max, ChatGPT plans) can't be used for tasks: providers allow automated use with API keys.
131+- Subscriptions (Claude Pro or Max, ChatGPT plans) can't be used for headless tasks: providers allow automated use with API keys. You can hand a task to the CLI signed in to [your own web terminal](/docs/agents/terminal#give-it-a-task) instead.
132132
133+## Usage and cost
134+
135+Each task's page has a **Usage** box: per run, the model requests, input and output tokens, cache reads and writes, and the cost, with the task's total. The same numbers are in the REST API (`usage` on the task, and on each of its `runs`) and in MCP's `agent_task`.
136+
137+- **Through your key**, Diffinery's model proxy counts the tokens from each answer the provider sends (`"source": "proxy"`). These are the numbers budgets and limits use.
138+- **In your terminal**, on a subscription, no request passes Diffinery, so the run shows what the CLI reported: Claude Code at the end of its run, Codex after each turn (`"source": "runtime"`).
139+- **Cost** is only ever Claude Code's own figure (`total_cost_usd`), at API prices. Diffinery keeps no price list. Codex and custom commands show tokens without a cost, and a subscription covers a terminal run's cost. Your provider's console shows what you were actually billed.
140+
141+**Settings → Model access** shows **Usage in the last 30 days** for every run you pay for: your tasks and your coordinator's, and [agent reviews](/docs/agents/review) in repositories where you turned them on. It is broken down by repository and kind, with each repository named as it was when its runs ended. A repository renamed since shows a line for each name. `GET /api/v3/diffinery/model-usage?days=30` returns the same (1 to 90 days); a token limited to some repositories sees only those.
142+
143+Model requests are limited to **600 a minute per paying person**, across all their runs, on top of each run's own limits. Past that, the proxy answers `429` with `retry-after: 60`, and Claude Code and Codex wait and retry. This caps how fast a runaway task, or a leaked run token, can spend your key. Like the other [rate limits](/docs/reference/rest-api#rate-limits), it is counted in each Cloudflare location, so runs in several locations can go past 600 a minute in total; each run's own limits and budgets still apply.
144+
133145 ## How your key is protected
134146
135147 Your key never enters a sandbox:
@@ -179,6 +191,7 @@
179191 | `POST .../tasks/{number}/stop` | Stop | The person who started it, its agent or a repository admin |
180192 | `GET /diffinery/tasks` | Tasks you started, across repositories | Any token |
181193 | `GET /diffinery/model-keys` | Your saved keys (provider, last four characters, dates) | Any token |
194+| `GET /diffinery/model-usage?days=30` | The model usage you paid for, per repository and kind (`task`, `review`, `coordinator`), over 1 to 90 days. See [Usage and cost](#usage-and-cost). | Any token; a token limited to some repositories sees only theirs |
182195 | `PUT` / `DELETE /diffinery/model-keys/{provider}` | Save or delete a key | Browser session only |
183196
184197 ```sh
@@ -206,11 +219,12 @@
206219 "queued_message": null,
207220 "error": null,
208221 "html_url": "https://diffinery.com/acme/website/tasks/7",
209- "runs": [{ "number": 1, "kind": "start", "message": null, "state": "done", "exit_code": 0, "error": null, "model_requests": 42, "tokens": 318204, "started_at": "2026-10-14T07:02:11.000Z", "ended_at": "2026-10-14T07:09:40.000Z" }]
222+ "usage": { "requests": 42, "input_tokens": 9120, "output_tokens": 14406, "cache_read_tokens": 2853120, "cache_write_tokens": 9366, "cost_usd": 1.4123 },
223+ "runs": [{ "number": 1, "kind": "start", "message": null, "state": "done", "exit_code": 0, "error": null, "model_requests": 42, "tokens": 318204, "usage": { "source": "proxy", "requests": 42, "input_tokens": 9120, "output_tokens": 14406, "cache_read_tokens": 2853120, "cache_write_tokens": 9366, "cost_usd": 1.4123 }, "started_at": "2026-10-14T07:02:11.000Z", "ended_at": "2026-10-14T07:09:40.000Z" }]
210224 }
211225 ```
212226
213-A run's `model_requests` and `tokens` are what its agent used through the model proxy, counted from the usage the provider reports (cache reads count as a tenth). An answer that reports no usage counts an estimate: a token per 3 bytes of request, plus the output it allows.
227+A run's `model_requests` and `tokens` are what its agent used through the model proxy, counted from the usage the provider reports (cache reads count as a tenth). An answer that reports no usage counts an estimate: a token per 3 bytes of request, plus the output it allows. `usage` breaks a run's tokens down, and says who counted them (`source`): `proxy`, or `runtime` for a run in your terminal. It is `null` for a run that used none, and the task's `usage` adds up its runs. See [Usage and cost](#usage-and-cost).
214228
215229 Transcript lines have `seq`, `run`, `at`, `stream` (`out` and `err` for the runtime's output, `setup` for installing and pushing, `note` for Diffinery's own lines) and `text`. Claude Code writes `stream-json` lines and Codex writes `--json` lines to `out`.
216230
modifiedapps/diffinery/docs/agents/terminal.md+1−1View file
@@ -53,7 +53,7 @@
5353
5454 Only your own click starts a run in your terminal. Nothing else does: not a coordinator, a schedule, a retry, [Best of N](/docs/agents/tasks#best-of-n) or a failing check. A task whose last run was in a terminal doesn't start a fix run when its checks fail; run it again yourself. A terminal runs one task at a time, it counts as attached while it does (so it doesn't end for being idle), and ending the terminal stops the task. Tasks that plan first or have more than one attempt run in sandboxes of their own.
5555
56-The task's transcript is kept, as for any task: it is the CLI's output for that run, not what you type in the terminal.
56+The task's transcript is kept, as for any task: it is the CLI's output for that run, not what you type in the terminal. Its **Usage** shows the tokens the CLI reported, and Claude Code's cost at API prices, which your subscription covers. See [Usage and cost](/docs/agents/tasks#usage-and-cost).
5757
5858 ### What a task can reach
5959
modifiedapps/diffinery/docs/reference/limits.md+2−0View file
@@ -275,6 +275,8 @@
275275 | Project instructions | 10,000 characters | ``422 `instructions` is longer than 10000 characters``. |
276276 | Coordinator | 5 tasks per post, 20 per person and project per 24 hours | A coordinator run lasts at most 30 minutes and 200 model requests, within its token budget (1M by default); each run of a task it starts has its own budget (2M by default). Both budgets are 100,000 to 50,000,000 tokens. See [Coordinator](/docs/agents/projects#coordinator). |
277277 | Project conversation post | 20,000 characters | ``422 `body` is longer than 20000 characters``. The conversation API and `project_posts` return at most 100 posts per call. |
278+| Model requests per person | 600 a minute, across all the runs you pay for | The model proxy answers `429` with `retry-after: 60`; the run goes on, and Claude Code and Codex retry. Counted in each Cloudflare location, like the [rate limits](/docs/reference/rest-api#rate-limits), so it isn't a global total. |
279+| Usage report | 1 to 90 days, 200 lines | `GET /diffinery/model-usage` and **Settings → Model access**. See [Usage and cost](/docs/agents/tasks#usage-and-cost). |
278280
279281 See [Agent tasks](/docs/agents/tasks). For the Today view's limits (repositories active in the last 30 days, at most 100 of them, 50 open pull requests each), see [Today](/docs/agents/today).
280282
modifiedapps/diffinery/docs/reference/rest-api.md+1−0View file
@@ -408,6 +408,7 @@
408408 | Anything else without signing in (web pages, API, git) | 300 a minute | IP address |
409409 | Anything with a session or token (web, API, MCP, git) | 2,000 a minute | Account |
410410 | Creating repositories, organizations, forks, issues, pull requests, releases, comments, reviews and reports, and MCP tools that change something | 60 a minute | Account |
411+| Agents' model requests through the model proxy (`/-/model/*`) | 600 a minute | The person who pays for the run |
411412
412413 Workflow jobs, sandboxes, self-hosted runners and imports use their own credentials and aren't counted. The counts are kept in each Cloudflare location, so they stop bursts rather than meter use exactly. Past a limit, the request answers `429` with a message naming the limit, `retry-after: 60` and `x-ratelimit-remaining: 0`; Octokit's throttling plugin waits and retries.
413414
modifiedapps/diffinery/docs/self-hosting/configuration.md+1−1View file
@@ -176,7 +176,7 @@
176176 | `MAIL` | Queue producer, with a consumer on the same Worker | `diffinery-mail`, dead-letter queue `diffinery-mail-dlq` | Email jobs. Consumer: batches of up to 20, waiting up to 2 seconds, 8 retries. Optional: without it emails are dropped with a warning in the logs. |
177177 | `EMAIL` | `send_email` (Email Sending) | Your onboarded domain | Sending email. Optional: without it every email is logged as `skipped`. |
178178 | (none) | Queue consumer | `diffinery-mail-events` | Bounces and complaints from Email Sending's event subscriptions, which you subscribe this queue to (see [Site administration](/docs/admin/site-administration#bounces-complaints-and-the-suppression-list)). Consumer: batches of up to 50, waiting up to 5 seconds, 8 retries. |
179-| `RL_AUTH`, `RL_ANON`, `RL_ACCOUNT`, `RL_CREATE` | Rate limiting (`ratelimits`) | Namespaces `1001` to `1004`; 20, 300, 2,000 and 60 requests per 60 seconds | [Request rate limits](/docs/reference/rest-api#rate-limits): sign-in per IP address, anonymous requests per IP address, signed-in requests per account, and new content per account. Optional: a missing binding (the `dev` environment declares none) or one that fails lets requests through. If you change a limit, change `RATE_LIMITS` in `src/lib/rate-limit.ts` to match, since the 429 messages quote it. Namespace IDs are shared across your Cloudflare account, so pick others if another Worker already uses these. |
179+| `RL_AUTH`, `RL_ANON`, `RL_ACCOUNT`, `RL_CREATE`, `RL_MODEL` | Rate limiting (`ratelimits`) | Namespaces `1001` to `1005`; 20, 300, 2,000, 60 and 600 requests per 60 seconds | [Request rate limits](/docs/reference/rest-api#rate-limits): sign-in per IP address, anonymous requests per IP address, signed-in requests per account, new content per account, and agents' model requests per paying person (`MODEL_RATE_LIMIT` in `src/lib/rate-limit.ts`). Optional: a missing binding (the `dev` environment declares none) or one that fails lets requests through. If you change a limit, change `RATE_LIMITS` (or `MODEL_RATE_LIMIT`) in `src/lib/rate-limit.ts` to match, since the 429 messages quote it. Namespace IDs are shared across your Cloudflare account, so pick others if another Worker already uses these. |
180180 | `REPO` | Durable Object namespace | Class `RepoDO` (SQLite) | One object per repository. |
181181 | `SCHEDULER` | Durable Object namespace | Class `SchedulerDO` (SQLite) | One object that starts scheduled workflow runs on its own alarm. |
182182 | `RUNNER` | Durable Object namespace with a container | Class `RunnerContainer` (SQLite) | One container per hosted workflow job. |
addedapps/diffinery/migrations/0030_agent_usage.sql+26−0View file
@@ -1,0 +1,26 @@
1+-- Model usage of each agent run that used any (phase 5a), written by the repository's ProjectDO when the run
2+-- ends, for the 30-day view in Settings → Model access. `person_id` is who pays: the task's person, or for an
3+-- agent review the admin who turned reviews on. `source` is `proxy` (counted by Diffinery from the provider's
4+-- answers) or `runtime` (what the CLI reported, in a terminal on a subscription); `cost_usd` is only ever the
5+-- CLI's own figure. `repository` is the repository's owner/name when the run ended, shown as it was then: the
6+-- payer may have lost access since, and a later name could reveal what they can no longer read.
7+CREATE TABLE agent_usage (
8+ repo_id TEXT NOT NULL,
9+ repository TEXT NOT NULL,
10+ task INTEGER NOT NULL,
11+ run INTEGER NOT NULL,
12+ person_id TEXT NOT NULL,
13+ kind TEXT NOT NULL,
14+ runtime TEXT NOT NULL,
15+ providers TEXT NOT NULL,
16+ source TEXT NOT NULL,
17+ requests INTEGER NOT NULL,
18+ input_tokens INTEGER NOT NULL,
19+ output_tokens INTEGER NOT NULL,
20+ cache_read_tokens INTEGER NOT NULL,
21+ cache_write_tokens INTEGER NOT NULL,
22+ cost_usd REAL,
23+ ended_at TEXT NOT NULL,
24+ PRIMARY KEY (repo_id, task, run)
25+);
26+CREATE INDEX agent_usage_person ON agent_usage(person_id, ended_at);
modifiedapps/diffinery/src/accounts/lifecycle.ts+1−0View file
@@ -106,6 +106,7 @@
106106 forMe("DELETE FROM account_links WHERE account_id = ?"),
107107 forIds("DELETE FROM model_keys WHERE account_id IN (?)"),
108108 forMe("DELETE FROM terminal_terms WHERE account_id = ?"),
109+ forMe("DELETE FROM agent_usage WHERE person_id = ?"),
109110 forIds("DELETE FROM account_quotas WHERE account_id IN (?)"),
110111 forIds("DELETE FROM compute_usage WHERE account_id IN (?)"),
111112 forIds("DELETE FROM login_redirects WHERE account_id IN (?)"),
modifiedapps/diffinery/src/agentic/ProjectDO.ts+58−6View file
@@ -3,13 +3,14 @@
33 import { localTimer } from "../auth/dev.js";
44 import type { Env } from "../env.js";
55 import { Directory } from "../lib/directory.js";
6+import { MODEL_RATE_LIMIT, modelRateLimited } from "../lib/rate-limit.js";
67 import { HttpError, now, randomToken, sha256 } from "../lib/util.js";
78 import { atLeast } from "../repo/types.js";
89 import { taskCaller, TaskAccessError } from "./access.js";
910 import { endTerminal, TERMINAL_IDLE_MS, TERMINAL_MAX_MS, terminalsEnabled, Terminals, type TerminalRow } from "./terminals.js";
1011 import { PROVIDERS, type Provider } from "./keys.js";
1112 import { enqueueMail } from "../mail/send.js";
12-import { formatTokens, pinRefusal, weighted, type Usage } from "./usage.js";
13+import { formatTokens, pinRefusal, reportedUsage, runUsage, weighted, type RunUsage, type Usage } from "./usage.js";
1314 import { attemptBranch, isPlanFirst, lineDiff, MAX_ATTEMPTS, MAX_CHECK_FIXES, MAX_PLAN, MAX_POST, MAX_RUNNING_DEFAULT, MEMORY_VERSIONS, projectContext, type MemoryRow, type PickState, type PlanState, type PostRow, type ProjectSettings } from "./project.js";
1415 import { COORDINATOR_MAX_MODEL_CALLS, COORDINATOR_TIMEOUT_MS, COORDINATOR_TOKENS_DEFAULT, coordinatorPrompt, parseActions, TASKS_PER_DAY, TASKS_PER_RUN, type CoordinatorSettings } from "./coordinator.js";
1516 import { countText, dayAgo, findingComment, rulesFiles, findingId, MAX_AUTOFIXES, placeFindings, REPO_DAILY_DEFAULT, REVIEW_APP, REVIEW_CHECK, REVIEW_CREATOR, REVIEW_MAX_MODEL_CALLS, REVIEW_TIMEOUT_MS, REVIEW_TOKENS_DEFAULT, reviewBody, reviewLimit, reviewOutput, reviewPrompt, fixGoal, shouldMailLimit, type Finding, type PriorFinding, type ReviewLimit, type ReviewSettings } from "./review.js";
@@ -128,6 +129,12 @@
128129 output_tokens: number;
129130 cache_read_tokens: number;
130131 cache_write_tokens: number;
132+ /** Phase 5a: what the CLI itself reported (a terminal's subscription, where no request passes the proxy), and its cost. */
133+ reported_input: number;
134+ reported_output: number;
135+ reported_cache_read: number;
136+ reported_cache_write: number;
137+ cost_usd: number | null;
131138 exit_code: number | null;
132139 error: string | null;
133140 checked_at: string | null;
@@ -158,7 +165,7 @@
158165 }
159166
160167 export interface TaskView extends TaskRow {
161- runs: Array<Pick<RunRow, "seq" | "kind" | "message" | "step" | "exit_code" | "error" | "model_calls" | "tokens" | "started_at" | "ended_at" | "attempt" | "terminal_id">>;
168+ runs: Array<Pick<RunRow, "seq" | "kind" | "message" | "step" | "exit_code" | "error" | "model_calls" | "tokens" | "started_at" | "ended_at" | "attempt" | "terminal_id"> & { usage: RunUsage | null }>;
162169 }
163170
164171 const SCHEMA = `
@@ -300,6 +307,11 @@
300307 ["tasks", "check_fix_sha", "check_fix_sha TEXT"],
301308 ["tasks", "land", "land INTEGER NOT NULL DEFAULT 0"],
302309 ["runs", "terminal_id", "terminal_id TEXT"],
310+ ["runs", "reported_input", "reported_input INTEGER NOT NULL DEFAULT 0"],
311+ ["runs", "reported_output", "reported_output INTEGER NOT NULL DEFAULT 0"],
312+ ["runs", "reported_cache_read", "reported_cache_read INTEGER NOT NULL DEFAULT 0"],
313+ ["runs", "reported_cache_write", "reported_cache_write INTEGER NOT NULL DEFAULT 0"],
314+ ["runs", "cost_usd", "cost_usd REAL"],
303315 ];
304316
305317 /** A finding as stored. */
@@ -327,6 +339,8 @@
327339 const TERMINAL_POLL_MS = 60_000;
328340 /** How often the person's access is re-checked while a run is active. */
329341 const RECHECK_MS = 60_000;
342+/** The run columns runUsage (usage.ts) reads. */
343+const USAGE_COLUMNS = "input_tokens, output_tokens, cache_read_tokens, cache_write_tokens, reported_input, reported_output, reported_cache_read, reported_cache_write, cost_usd";
330344 const MAX_LINE = 32 * 1024;
331345 /** A review's findings arrive in one line of output, so its lines may be longer. */
332346 const MAX_REVIEW_LINE = 512 * 1024;
@@ -412,8 +426,8 @@
412426 get(number: number): TaskView | null {
413427 const t = this.task(number);
414428 if (!t) return null;
415- const runs = this.sql.exec<RunRow>("SELECT seq, kind, message, step, exit_code, error, model_calls, tokens, started_at, ended_at, attempt, terminal_id FROM runs WHERE task = ? ORDER BY seq", number).toArray();
416- return { ...t, runs };
429+ const runs = this.sql.exec<RunRow>(`SELECT seq, kind, message, step, exit_code, error, model_calls, tokens, started_at, ended_at, attempt, terminal_id, ${USAGE_COLUMNS} FROM runs WHERE task = ? ORDER BY seq`, number).toArray();
430+ return { ...t, runs: runs.map((r) => ({ seq: r.seq, kind: r.kind, message: r.message, step: r.step, exit_code: r.exit_code, error: r.error, model_calls: r.model_calls, tokens: r.tokens, started_at: r.started_at, ended_at: r.ended_at, attempt: r.attempt, terminal_id: r.terminal_id, usage: runUsage(r) })) };
417431 }
418432
419433 lines(number: number, after = 0, limit = 2000): TranscriptLine[] {
@@ -985,7 +999,7 @@
985999 async modelAuth(
9861000 token: string,
9871001 req: { provider: Provider; estimate: Usage | null; reviewRefuse?: string; model?: string | null; routing?: string },
988- ): Promise<{ task: number; run: number; personId: string; agentId: string; repoId: string; bounded: boolean } | { limit: string } | { refuse: string } | null> {
1002+ ): Promise<{ task: number; run: number; personId: string; agentId: string; repoId: string; bounded: boolean } | { limit: string } | { refuse: string } | { busy: string } | null> {
9891003 const hash = await sha256(token);
9901004 const r = this.sql.exec<RunRow>("SELECT * FROM runs WHERE token_hash = ? AND step = 'agent'", hash).toArray()[0];
9911005 if (!r) return null;
@@ -995,6 +1009,8 @@
9951009 if (bound && req.reviewRefuse) return { refuse: req.reviewRefuse };
9961010 const pinned = pinRefusal(req, { review: !!bound, model: t.model });
9971011 if (pinned) return { refuse: pinned };
1012+ // Phase 5a: the paying person's rate across all their runs. Nothing is booked or counted, and the run goes on: the CLI retries.
1013+ if (await modelRateLimited(this.env, t.person_id)) return { busy: `Too many model requests from ${t.person_login}'s agents: at most ${MODEL_RATE_LIMIT} a minute. Try again in a minute.` };
9981014 const calls = r.model_calls + 1;
9991015 const cost = req.estimate ? weighted(req.estimate) : 0;
10001016 if (bound) {
@@ -1047,6 +1063,8 @@
10471063 task,
10481064 run,
10491065 );
1066+ // An answer settled after its run ended still reaches the run's usage row.
1067+ if (this.run(task, run)?.ended_at) await this.ledger(task, run);
10501068 }
10511069
10521070 // ---------------------------------------------------------------- runs
@@ -1775,7 +1793,12 @@
17751793 const p = (await this.sandbox(r, "/proc/read", { id: r.proc, stdoutOffset: r.out_off, stderrOffset: r.err_off })) as { running: boolean; exitCode: number | null; timedOut: boolean; stdout: string; stderr: string; stdoutOffset: number; stderrOffset: number };
17761794 // A review's findings, or a coordinator's actions, arrive in one long line.
17771795 const max = this.task(r.task)?.kind === "task" ? MAX_LINE : MAX_REVIEW_LINE;
1778- for (const l of splitLines(p.stdout)) this.line(r.task, r.seq, out, l, max);
1796+ const runtime = out === "out" ? this.task(r.task)?.runtime : undefined;
1797+ for (const l of splitLines(p.stdout)) {
1798+ // Usage the CLI reports, read from the whole line before the transcript cuts it.
1799+ if (runtime) this.reported(r, runtime, l);
1800+ this.line(r.task, r.seq, out, l, max);
1801+ }
17791802 for (const l of splitLines(p.stderr)) this.line(r.task, r.seq, out === "out" ? "err" : "setup", l);
17801803 this.sql.exec("UPDATE runs SET out_off = ?, err_off = ? WHERE task = ? AND seq = ?", p.stdoutOffset, p.stderrOffset, r.task, r.seq);
17811804 // Output read while it was running, or more than one read's worth after it ended: read again on the next tick.
@@ -1860,6 +1883,7 @@
18601883 } else if (!queued) settle();
18611884 // Today (D1) shows the settled state now too, not once the sandbox has closed.
18621885 await this.index(r.task);
1886+ await this.ledger(r.task, r.seq);
18631887 if (r.sandbox_id) {
18641888 if (r.proc) await this.sandbox(r, "/proc/kill", { id: r.proc, signal: "SIGKILL" }).catch(() => {});
18651889 // A terminal's sandbox stays the person's: only the run's own checkout goes.
@@ -1938,6 +1962,16 @@
19381962 return this.sql.exec<PostRow>("SELECT * FROM posts WHERE n = last_insert_rowid()").one();
19391963 }
19401964
1965+ /** Phase 5a: keep what a Claude Code or Codex run reports about its own usage (usage.ts reportedUsage). */
1966+ private reported(r: RunRow, runtime: string, line: string): void {
1967+ const rep = reportedUsage(runtime, line);
1968+ if (!rep) return;
1969+ const { usage: u, costUsd, mode } = rep;
1970+ if (mode === "set")
1971+ this.sql.exec("UPDATE runs SET reported_input = ?, reported_output = ?, reported_cache_read = ?, reported_cache_write = ?, cost_usd = ? WHERE task = ? AND seq = ?", u.input, u.output, u.cacheRead, u.cacheWrite, costUsd, r.task, r.seq);
1972+ else this.sql.exec("UPDATE runs SET reported_input = reported_input + ?, reported_output = reported_output + ?, reported_cache_read = reported_cache_read + ?, reported_cache_write = reported_cache_write + ? WHERE task = ? AND seq = ?", u.input, u.output, u.cacheRead, u.cacheWrite, r.task, r.seq);
1973+ }
1974+
19411975 private note(task: number, run: number, text: string): void {
19421976 this.line(task, run, "note", text);
19431977 }
@@ -1949,6 +1983,24 @@
19491983 this.sql.exec("INSERT INTO lines (task, seq, run, at, stream, text) VALUES (?, ?, ?, ?, ?, ?)", task, seq, run, now(), stream, text.length > max ? `${text.slice(0, max)} … (line cut at ${max} characters)` : text);
19501984 }
19511985
1986+ /** Phase 5a: an ended run's usage in D1 (agent_usage), for its payer's 30-day view in Settings → Model access. */
1987+ private async ledger(task: number, seq: number): Promise<void> {
1988+ const t = this.task(task);
1989+ const r = this.run(task, seq);
1990+ if (!t || !r?.ended_at) return;
1991+ const u = runUsage(r);
1992+ if (!u) return;
1993+ const providers = r.terminal_id ? "subscription" : (JSON.parse(r.providers || "[]") as string[]).join(",");
1994+ // The repository's name as of now, kept on the row: a later rename or transfer never shows to someone who lost access (Security, 5a).
1995+ const repo = await new Directory(this.env).repoById(t.repo_id);
1996+ await this.env.DB.prepare(
1997+ "INSERT INTO agent_usage (repo_id, repository, task, run, person_id, kind, runtime, providers, source, requests, input_tokens, output_tokens, cache_read_tokens, cache_write_tokens, cost_usd, ended_at) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) ON CONFLICT (repo_id, task, run) DO UPDATE SET source = excluded.source, requests = excluded.requests, input_tokens = excluded.input_tokens, output_tokens = excluded.output_tokens, cache_read_tokens = excluded.cache_read_tokens, cache_write_tokens = excluded.cache_write_tokens, cost_usd = excluded.cost_usd",
1998+ )
1999+ .bind(t.repo_id, repo ? `${repo.owner_login}/${repo.name}` : "", task, seq, t.person_id, t.kind, t.runtime, providers, u.source, u.requests, u.input, u.output, u.cacheRead, u.cacheWrite, u.costUsd, r.ended_at)
2000+ .run()
2001+ .catch((e) => console.error("agent_usage failed", e));
2002+ }
2003+
19522004 /** The D1 index of tasks across repositories, for Today and a person's task list. */
19532005 private async index(number: number): Promise<void> {
19542006 const t = this.task(number);
modifiedapps/diffinery/src/agentic/api.ts+10−0View file
@@ -7,6 +7,7 @@
77 import { approvePlan, attemptJson, compareAttempts, getTask, pickAttempt, setTaskOptions, lineJson, messageTask, myTasks, projectStub, requirePersonSession, requireTaskReader, retryTask, startTask, stopTask, taskJson, taskNumber, type StartInput } from "./tasks.js";
88 import { askForReview, findingJson, reviewSettings, reviewSettingsJson, saveReviewSettings, type SettingsInput } from "./reviews.js";
99 import { gatherToday, todayJson } from "./today.js";
10+import { usageDays, usageReport, usageReportJson } from "./ledger.js";
1011 import { boardJson, coordinatorJson, handlePost, splitPlan, loadBoard, memoryJson, postJson, postToProject, saveCoordinator, saveMemory, saveProjectSettings, settingsJson, type CoordinatorInput } from "./projects.js";
1112
1213 async function jsonBody<T>(c: AppContext): Promise<T> {
@@ -38,6 +39,15 @@
3839
3940 agenticApi.get("/diffinery/model-keys", async (c) => c.json((await new ModelKeys(c.env).list(requireAccount(c).id)).map(keyJson)));
4041
42+/** Phase 5a: the model usage the caller paid for, across repositories, over the last `days` (1 to 90, default 30). */
43+agenticApi.get("/diffinery/model-usage", async (c) => {
44+ const report = await usageReport(c.env, requireAccount(c).id, usageDays(c.req.query("days")));
45+ // A token limited to some repositories sees only their lines.
46+ const only = tokenRepoIds(c);
47+ if (only) report.lines = report.lines.filter((l) => only.includes(l.repoId));
48+ return c.json(usageReportJson(report));
49+});
50+
4151 agenticApi.put("/diffinery/model-keys/:provider", async (c) => {
4252 // A key is spent by tasks this person starts; only they, in a browser, can set it.
4353 const me = requirePersonSession(c);
addedapps/diffinery/src/agentic/ledger.ts+89−0View file
@@ -1,0 +1,89 @@
1+import type { Env } from "../env.js";
2+
3+/**
4+ * A person's model usage across repositories (phase 5a), from the `agent_usage` ledger each repository's
5+ * ProjectDO writes when a run ends: for Settings → Model access and `GET /diffinery/model-usage`. Only the
6+ * payer's own runs: their tasks, their coordinator's, and reviews in repositories where they turned them on.
7+ * Repositories are named as they were when each run ended, never by a later name the payer may not be able to read.
8+ */
9+
10+export const USAGE_DAYS_DEFAULT = 30;
11+export const USAGE_DAYS_MAX = 90;
12+
13+export interface UsageLine {
14+ repoId: string;
15+ /** owner/name when the runs ended (null if unknown then); a renamed repository shows a line per name. */
16+ repository: string | null;
17+ kind: "task" | "review" | "coordinator";
18+ runs: number;
19+ requests: number;
20+ input: number;
21+ output: number;
22+ cacheRead: number;
23+ cacheWrite: number;
24+ /** Only Claude Code's own figures, added up; null when no run reported one. */
25+ costUsd: number | null;
26+ /** Runs counted from what the CLI in a terminal reported, not by the model proxy. */
27+ reportedRuns: number;
28+}
29+
30+export interface UsageReport {
31+ days: number;
32+ since: string;
33+ lines: UsageLine[];
34+}
35+
36+export const usageDays = (v: unknown): number => {
37+ const d = Math.floor(Number(v ?? USAGE_DAYS_DEFAULT));
38+ return Number.isFinite(d) ? Math.min(Math.max(d, 1), USAGE_DAYS_MAX) : USAGE_DAYS_DEFAULT;
39+};
40+
41+export async function usageReport(env: Env, personId: string, days = USAGE_DAYS_DEFAULT): Promise<UsageReport> {
42+ const since = new Date(Date.now() - days * 86400_000).toISOString();
43+ const { results } = await env.DB.prepare(
44+ `SELECT u.repo_id, u.repository, u.kind, count(*) AS runs, sum(u.requests) AS requests, sum(u.input_tokens) AS input,
45+ sum(u.output_tokens) AS output, sum(u.cache_read_tokens) AS cache_read, sum(u.cache_write_tokens) AS cache_write,
46+ sum(u.cost_usd) AS cost, sum(u.source = 'runtime') AS reported
47+ FROM agent_usage u
48+ WHERE u.person_id = ? AND u.ended_at >= ?
49+ GROUP BY u.repo_id, u.repository, u.kind
50+ ORDER BY sum(u.input_tokens + u.output_tokens + u.cache_read_tokens + u.cache_write_tokens) DESC
51+ LIMIT 200`,
52+ )
53+ .bind(personId, since)
54+ .all<{ repo_id: string; repository: string; kind: UsageLine["kind"]; runs: number; requests: number; input: number; output: number; cache_read: number; cache_write: number; cost: number | null; reported: number }>();
55+ return {
56+ days,
57+ since,
58+ lines: results.map((x) => ({
59+ repoId: x.repo_id,
60+ repository: x.repository || null,
61+ kind: x.kind,
62+ runs: x.runs,
63+ requests: x.requests,
64+ input: x.input,
65+ output: x.output,
66+ cacheRead: x.cache_read,
67+ cacheWrite: x.cache_write,
68+ costUsd: x.cost,
69+ reportedRuns: x.reported,
70+ })),
71+ };
72+}
73+
74+export const usageReportJson = (r: UsageReport) => ({
75+ days: r.days,
76+ since: r.since,
77+ repositories: r.lines.map((l) => ({
78+ repository: l.repository,
79+ kind: l.kind,
80+ runs: l.runs,
81+ reported_by_cli_runs: l.reportedRuns,
82+ requests: l.requests,
83+ input_tokens: l.input,
84+ output_tokens: l.output,
85+ cache_read_tokens: l.cacheRead,
86+ cache_write_tokens: l.cacheWrite,
87+ cost_usd: l.costUsd === null ? null : Math.round(l.costUsd * 10_000) / 10_000,
88+ })),
89+});
modifiedapps/diffinery/src/agentic/proxy.ts+5−0View file
@@ -95,6 +95,11 @@
9595 if (!run) return err(401, "this model token isn't valid: its task run has ended");
9696 if ("limit" in run) return err(429, run.limit);
9797 if ("refuse" in run) return err(403, run.refuse);
98+ if ("busy" in run) {
99+ const res = err(429, run.busy);
100+ res.headers.set("retry-after", "60");
101+ return res;
102+ }
98103 // Runs with a budget (reviews, coordinators and the tasks they start) send the capped, tool-stripped body their booking assumes.
99104 if (run.bounded && prepared.reviewBody) body = prepared.reviewBody;
100105 // A booked request that never reaches the provider, or that it answers with an error, costs nothing.
modifiedapps/diffinery/src/agentic/tasks-web.tsx+147−1View file
@@ -11,11 +11,12 @@
1111 import type { TaskRow, TaskView } from "./ProjectDO.js";
1212 import { fixGoal } from "./review.js";
1313 import { MAX_ATTEMPTS, MAX_CHECK_FIXES } from "./project.js";
14-import { formatTokens } from "./usage.js";
14+import { formatCost, formatTokens, totalUsage, type RunUsage } from "./usage.js";
1515 import { OPENROUTER_DEFAULT_MODEL, parseRuntimeChoice, runtimeChoices, runtimeLabel, transcriptItem, type TranscriptItem } from "./runtimes.js";
1616 import { approvePlan, compareAttempts, drivableAgents, getTask, pickAttempt, setTaskOptions, type AttemptCompare, mayStop, messageTask, projectStub, requireTaskReader, retryTask, startTask, stopTask, taskNumber } from "./tasks.js";
1717 import { splitPlan } from "./projects.js";
1818 import { runTaskInTerminal, terminalForTask, terminalId } from "./terminals.js";
19+import { usageReport, type UsageLine, type UsageReport } from "./ledger.js";
1920
2021 /**
2122 * Web pages for agent tasks: Settings → Model access, and a repository's
@@ -49,12 +50,91 @@
4950
5051 export const TaskBadge: FC<{ state: TaskRow["state"] }> = ({ state }) => <span class={`badge ${TASK_STATE[state].tone}`}>{TASK_STATE[state].label}</span>;
5152
53+// ---------------------------------------------------------------- usage (phase 5a)
54+
55+const RUN_KIND: Record<TaskView["runs"][number]["kind"], string> = { start: "Start", message: "Follow-up", retry: "Retry", review: "Review", coordinate: "Coordinate", plan: "Plan", build: "Build", fix: "Fix checks" };
56+
57+const UsageCells: FC<{ u: Omit<RunUsage, "source"> | null; subscription?: boolean }> = ({ u, subscription }) =>
58+ u ? (
59+ <>
60+ <td class="r">{u.requests ? u.requests.toLocaleString("en-US") : "—"}</td>
61+ <td class="r">{formatTokens(u.input)}</td>
62+ <td class="r">{formatTokens(u.output)}</td>
63+ <td class="r">
64+ {formatTokens(u.cacheRead)} / {formatTokens(u.cacheWrite)}
65+ </td>
66+ <td class="r">{u.costUsd === null ? "—" : `${formatCost(u.costUsd)}${subscription ? "*" : ""}`}</td>
67+ </>
68+ ) : (
69+ <td colspan={5} class="muted">
70+ None reported
71+ </td>
72+ );
73+
74+/** A task's model usage, per run and in all: counted by Diffinery's model proxy, or as the CLI in a terminal reported it. */
75+const UsageBox: FC<{ t: TaskView }> = ({ t }) => {
76+ const total = totalUsage(t.runs.map((r) => r.usage));
77+ if (!total) return null;
78+ const subscription = t.runs.some((r) => r.usage?.source === "runtime" && r.usage.costUsd !== null);
79+ return (
80+ <details class="box" id="usage">
81+ <summary class="head">
82+ <h2>Usage</h2>
83+ <span class="badge">
84+ {formatTokens(total.input + total.output + total.cacheRead + total.cacheWrite)} tokens{total.costUsd !== null ? ` · ${formatCost(total.costUsd)}` : ""}
85+ </span>
86+ </summary>
87+ <div class="body">
88+ <div class="xscroll">
89+ <table class="tbl">
90+ <thead>
91+ <tr>
92+ <th>Run</th>
93+ <th>Counted by</th>
94+ <th class="r">Requests</th>
95+ <th class="r">Input</th>
96+ <th class="r">Output</th>
97+ <th class="r">Cache read / write</th>
98+ <th class="r">Cost</th>
99+ </tr>
100+ </thead>
101+ <tbody>
102+ {t.runs.map((r) => (
103+ <tr>
104+ <td>
105+ {r.seq} · {RUN_KIND[r.kind]}
106+ {r.attempt ? ` · attempt ${r.attempt}` : ""}
107+ </td>
108+ <td class="muted">{r.usage ? (r.usage.source === "proxy" ? "Diffinery" : `${runtimeLabel(t.runtime)} in the terminal`) : "—"}</td>
109+ <UsageCells u={r.usage} subscription={r.usage?.source === "runtime"} />
110+ </tr>
111+ ))}
112+ {t.runs.length > 1 ? (
113+ <tr>
114+ <th>Total</th>
115+ <td />
116+ <UsageCells u={total} />
117+ </tr>
118+ ) : null}
119+ </tbody>
120+ </table>
121+ </div>
122+ <p class="muted small mt">
123+ Requests through your key are counted by Diffinery from the provider's answers. A run in your terminal is counted by the CLI signed in there. Cost is only ever Claude Code's own figure: Codex and custom commands report none.
124+ {subscription ? " * At API prices: a subscription covers it." : ""} Your usage across repositories is in <a href="/settings/model-access#usage">Settings → Model access</a>.
125+ </p>
126+ </div>
127+ </details>
128+ );
129+};
130+
52131 // ---------------------------------------------------------------- Settings → Model access
53132
54133 const ModelAccessPage = async (c: AppContext) => {
55134 const me = requireAccount(c);
56135 const t = csrfToken(c);
57136 const keys = new Map((await new ModelKeys(c.env).list(me.id)).map((k) => [k.provider, k]));
137+ const usage = await usageReport(c.env, me.id);
58138 return render(
59139 c,
60140 { title: "Model access" },
@@ -124,11 +204,76 @@
124204 </div>
125205 );
126206 })}
207+ <UsageReportBox report={usage} />
127208 </div>
128209 </div>,
129210 );
130211 };
131212
213+const USAGE_KIND: Record<UsageLine["kind"], string> = { task: "Tasks", review: "Agent reviews", coordinator: "Coordinator" };
214+
215+/** Phase 5a: what the person's agents used, per repository and kind, from the agent_usage ledger (ledger.ts). */
216+const UsageReportBox: FC<{ report: UsageReport }> = ({ report }) => {
217+ const total = report.lines.reduce(
218+ (a, l) => ({ runs: a.runs + l.runs, requests: a.requests + l.requests, input: a.input + l.input, output: a.output + l.output, cacheRead: a.cacheRead + l.cacheRead, cacheWrite: a.cacheWrite + l.cacheWrite, costUsd: l.costUsd === null ? a.costUsd : (a.costUsd ?? 0) + l.costUsd }),
219+ { runs: 0, requests: 0, input: 0, output: 0, cacheRead: 0, cacheWrite: 0, costUsd: null as number | null },
220+ );
221+ return (
222+ <div class="box" id="usage">
223+ <div class="head">
224+ <h2>Usage in the last {report.days} days</h2>
225+ {report.lines.length ? <span class="badge">{formatTokens(total.input + total.output + total.cacheRead + total.cacheWrite)} tokens</span> : null}
226+ </div>
227+ <div class="body">
228+ {report.lines.length ? (
229+ <div class="xscroll">
230+ <table class="tbl">
231+ <thead>
232+ <tr>
233+ <th>Repository</th>
234+ <th>For</th>
235+ <th class="r">Runs</th>
236+ <th class="r">Requests</th>
237+ <th class="r">Input</th>
238+ <th class="r">Output</th>
239+ <th class="r">Cache read / write</th>
240+ <th class="r">Cost</th>
241+ </tr>
242+ </thead>
243+ <tbody>
244+ {report.lines.map((l) => (
245+ <tr>
246+ <td>{l.repository ? <a href={`/${l.repository}/tasks`}>{l.repository}</a> : <span class="muted">Unknown</span>}</td>
247+ <td>{USAGE_KIND[l.kind]}</td>
248+ <td class="r">
249+ {l.runs}
250+ {l.reportedRuns ? <span class="muted"> ({l.reportedRuns} in a terminal)</span> : null}
251+ </td>
252+ <UsageCells u={l} />
253+ </tr>
254+ ))}
255+ {report.lines.length > 1 ? (
256+ <tr>
257+ <th>Total</th>
258+ <td />
259+ <td class="r">{total.runs}</td>
260+ <UsageCells u={total} />
261+ </tr>
262+ ) : null}
263+ </tbody>
264+ </table>
265+ </div>
266+ ) : (
267+ <p class="muted">None of your agents' runs used a model in the last {report.days} days.</p>
268+ )}
269+ <p class="muted small mt">
270+ Runs you pay for: your tasks and your coordinator's, and agent reviews in repositories where you turned them on. Repositories are named as they were when the runs ended. Diffinery counts requests through your keys from the provider's answers; runs in your terminal count what the CLI signed in there reported. Cost is only Claude Code's own figure, at API prices; check your provider's console for what you were billed. Also at <code>GET /api/v3/diffinery/model-usage</code>.
271+ </p>
272+ </div>
273+ </div>
274+ );
275+};
276+
132277 // ---------------------------------------------------------------- repository Tasks tab
133278
134279 async function forTasks(c: AppContext): Promise<Loaded> {
@@ -310,6 +455,7 @@
310455 </div>
311456 </Form>
312457 ) : null}
458+ <UsageBox t={t} />
313459 {t.queued ? <div class="flash info">A message is waiting for the current run to end: {t.queued}</div> : null}
314460 <div class="box">
315461 <div class="head">
modifiedapps/diffinery/src/agentic/tasks.ts+4−1View file
@@ -11,6 +11,7 @@
1111 import { MAX_ATTEMPTS } from "./project.js";
1212 import { checksState, type ChecksState } from "./today-repo.js";
1313 import { isRuntime, MAX_COMMAND, MAX_GOAL, OPENROUTER_DEFAULT_MODEL, OPENROUTER_MODEL, RUNTIMES, runtimeLabel, runtimeNeeds, type ModelRoute, type RuntimeId, type TranscriptLine } from "./runtimes.js";
14+import { totalUsage, usageJson } from "./usage.js";
1415
1516 /**
1617 * Who may do what with agent tasks. Shared by the REST API, MCP and the web.
@@ -323,10 +324,12 @@
323324 html_url: `${base}/${repoFull}/tasks/${t.number}`,
324325 created_at: t.created_at,
325326 updated_at: t.updated_at,
326- ...("runs" in t ? { runs: t.runs.map((r) => ({ number: r.seq, kind: r.kind, message: r.message, state: r.step, exit_code: r.exit_code, error: r.error, model_requests: r.model_calls ?? 0, tokens: r.tokens ?? 0, ...(r.attempt ? { attempt: r.attempt } : {}), ...(r.terminal_id ? { in_terminal: true } : {}), started_at: r.started_at, ended_at: r.ended_at })) } : {}),
327+ ...("runs" in t ? { usage: usageOrNull(totalUsage(t.runs.map((r) => r.usage))), runs: t.runs.map((r) => ({ number: r.seq, kind: r.kind, message: r.message, state: r.step, exit_code: r.exit_code, error: r.error, model_requests: r.model_calls ?? 0, tokens: r.tokens ?? 0, usage: usageOrNull(r.usage), ...(r.attempt ? { attempt: r.attempt } : {}), ...(r.terminal_id ? { in_terminal: true } : {}), started_at: r.started_at, ended_at: r.ended_at })) } : {}),
327328 };
328329 }
329330
331+const usageOrNull = (u: Parameters<typeof usageJson>[0] | null) => (u ? usageJson(u) : null);
332+
330333 export const lineJson = (l: TranscriptLine) => ({ seq: l.seq, run: l.run, at: l.at, stream: l.stream, text: l.text });
331334
332335 export interface MyTask {
modifiedapps/diffinery/src/agentic/usage.ts+85−0View file
@@ -95,6 +95,91 @@
9595
9696 export const formatTokens = (t: number): string => (t >= 1_000_000 ? `${+(t / 1_000_000).toFixed(1)}M` : t >= 1000 ? `${Math.round(t / 1000)}k` : String(t));
9797
98+/**
99+ * What a CLI reports about its own usage in one line of its output (phase 5a), for runs whose model
100+ * requests don't pass Diffinery: a terminal's signed-in subscription. Claude Code's `result` event
101+ * gives its run's totals and cost (`set`); each Codex `turn.completed` gives one turn's tokens (`add`).
102+ */
103+export function reportedUsage(runtime: string, line: string): { usage: Usage; costUsd: number | null; mode: "set" | "add" } | null {
104+ const claude = runtime === "claude-code" && line.startsWith('{"type":"result"');
105+ const codex = runtime === "codex" && line.startsWith('{"type":"turn.completed"');
106+ if (!claude && !codex) return null;
107+ const j = parse(line) as Record<string, unknown> | null;
108+ if (!j || typeof j !== "object") return null;
109+ const u = j.usage as Record<string, unknown> | undefined;
110+ if (!u || typeof u !== "object") return null;
111+ if (codex) {
112+ // Codex: input_tokens includes the cached ones.
113+ const cached = n(u.cached_input_tokens);
114+ return { usage: { input: Math.max(n(u.input_tokens) - cached, 0), output: n(u.output_tokens), cacheRead: cached, cacheWrite: 0 }, costUsd: null, mode: "add" };
115+ }
116+ const cost = typeof j.total_cost_usd === "number" && Number.isFinite(j.total_cost_usd) && j.total_cost_usd >= 0 ? j.total_cost_usd : null;
117+ return { usage: { input: n(u.input_tokens), output: n(u.output_tokens), cacheRead: n(u.cache_read_input_tokens), cacheWrite: n(u.cache_creation_input_tokens) }, costUsd: cost, mode: "set" };
118+}
119+
120+/**
121+ * A run's usage as people see it (phase 5a). Requests through the model proxy are counted by Diffinery
122+ * (`proxy`); a run with none, in a terminal on a subscription, has only what its CLI reported (`runtime`).
123+ * The cost is always the CLI's own figure (Claude Code's), never computed here.
124+ */
125+export interface RunUsage {
126+ source: "proxy" | "runtime";
127+ requests: number;
128+ input: number;
129+ output: number;
130+ cacheRead: number;
131+ cacheWrite: number;
132+ costUsd: number | null;
133+}
134+
135+export function runUsage(r: {
136+ model_calls: number;
137+ input_tokens: number;
138+ output_tokens: number;
139+ cache_read_tokens: number;
140+ cache_write_tokens: number;
141+ reported_input: number;
142+ reported_output: number;
143+ reported_cache_read: number;
144+ reported_cache_write: number;
145+ cost_usd: number | null;
146+}): RunUsage | null {
147+ const costUsd = r.cost_usd ?? null;
148+ if (r.model_calls > 0) return { source: "proxy", requests: r.model_calls, input: r.input_tokens, output: r.output_tokens, cacheRead: r.cache_read_tokens, cacheWrite: r.cache_write_tokens, costUsd };
149+ if (r.reported_input || r.reported_output || r.reported_cache_read || r.reported_cache_write || costUsd !== null)
150+ return { source: "runtime", requests: 0, input: r.reported_input, output: r.reported_output, cacheRead: r.reported_cache_read, cacheWrite: r.reported_cache_write, costUsd };
151+ return null;
152+}
153+
154+/** Several runs' usage added up; the cost counts the runs that reported one, and is null when none did. */
155+export function totalUsage(list: Array<RunUsage | null>): Omit<RunUsage, "source"> | null {
156+ const xs = list.filter((u): u is RunUsage => !!u);
157+ if (!xs.length) return null;
158+ const costs = xs.map((u) => u.costUsd).filter((c): c is number => c !== null);
159+ return {
160+ requests: xs.reduce((a, u) => a + u.requests, 0),
161+ input: xs.reduce((a, u) => a + u.input, 0),
162+ output: xs.reduce((a, u) => a + u.output, 0),
163+ cacheRead: xs.reduce((a, u) => a + u.cacheRead, 0),
164+ cacheWrite: xs.reduce((a, u) => a + u.cacheWrite, 0),
165+ costUsd: costs.length ? costs.reduce((a, c) => a + c, 0) : null,
166+ };
167+}
168+
169+/** Usage as the REST API and MCP give it, with GitHub-style snake_case names. */
170+export const usageJson = (u: Omit<RunUsage, "source"> & { source?: RunUsage["source"] }) => ({
171+ ...(u.source ? { source: u.source } : {}),
172+ requests: u.requests,
173+ input_tokens: u.input,
174+ output_tokens: u.output,
175+ cache_read_tokens: u.cacheRead,
176+ cache_write_tokens: u.cacheWrite,
177+ cost_usd: u.costUsd === null ? null : Math.round(u.costUsd * 10_000) / 10_000,
178+});
179+
180+/** A cost in dollars as a person reads it: cents, or less than a cent. */
181+export const formatCost = (usd: number): string => (usd > 0 && usd < 0.01 ? "<$0.01" : `$${usd.toFixed(2)}`);
182+
98183 /** The output an estimate assumes when a request doesn't cap it. */
99184 export const DEFAULT_MAX_OUTPUT = 32_000;
100185 /** What a compaction may write back, which its request can't cap: the most any current OpenAI model emits. */
modifiedapps/diffinery/src/env.ts+2−0View file
@@ -47,6 +47,8 @@
4747 RL_ANON?: RateLimit;
4848 RL_ACCOUNT?: RateLimit;
4949 RL_CREATE?: RateLimit;
50+ /** Model requests through the proxy, per paying person (agentic/proxy.ts). */
51+ RL_MODEL?: RateLimit;
5052 /** Email jobs, sent by the queue consumer in mail/send.ts. */
5153 MAIL?: Queue<MailJob>;
5254 IMPORT_WORKFLOW: Workflow<{ importId: string }>;
modifiedapps/diffinery/src/lib/rate-limit.ts+18−0View file
@@ -74,6 +74,24 @@
7474 return null;
7575 }
7676
77+/**
78+ * Model requests through the proxy (/-/model/*), per paying person across all their runs (phase 5a): above the
79+ * per-run limits and budgets, it caps how fast a leaked run token or a runaway task can spend a person's key. Kept
80+ * out of `bucketsFor` because only the run's ProjectDO knows who pays. Must match RL_MODEL in wrangler.jsonc.
81+ */
82+export const MODEL_RATE_LIMIT = 600;
83+
84+/** Whether this person is over their model request rate. A missing (local) or failing binding lets it through. */
85+export async function modelRateLimited(env: Env, personId: string): Promise<boolean> {
86+ if (!env.RL_MODEL) return false;
87+ try {
88+ return !(await env.RL_MODEL.limit({ key: `model:${personId}` })).success;
89+ } catch (e) {
90+ console.error("model rate limiter failed", e);
91+ return false;
92+ }
93+}
94+
7795 const WHAT: Record<Bucket, string> = {
7896 RL_AUTH: "sign-in attempts from your network",
7997 RL_ANON: "requests from your network without signing in",
modifiedapps/diffinery/src/web/assets.ts+2−0View file
@@ -278,6 +278,8 @@
278278 table.tbl{width:100%;border-collapse:collapse;font-size:13px}
279279 table.tbl th{text-align:left;font-weight:600;color:var(--fg2);padding:8px 12px;border-bottom:1px solid var(--border);background:var(--bg2);white-space:nowrap}
280280 table.tbl td{padding:8px 12px;border-top:1px solid var(--border2);vertical-align:middle}
281+table.tbl th.r,table.tbl td.r{text-align:right;font-variant-numeric:tabular-nums}
282+.xscroll{overflow-x:auto}
281283 .secret{font-family:var(--mono);background:var(--amber-soft);border:1px solid color-mix(in srgb,var(--amber) 45%,transparent);border-radius:var(--r);padding:7px 12px;line-height:20px;min-height:36px;word-break:break-all}
282284 .pager{display:flex;gap:8px;justify-content:center;margin:16px 0}
283285 .sha{font-family:var(--mono);font-size:12px;padding:1px 6px;border-radius:5px;background:var(--bg3);color:var(--fg)}
modifiedapps/diffinery/test/e2e/agents-harness.mjs+3−2View file
@@ -109,7 +109,7 @@
109109 req.on("end", () => {
110110 modelRequests.push({ method: req.method, url: req.url, headers: req.headers, body });
111111 res.writeHead(200, { "content-type": "application/json", "set-cookie": "upstream=1" });
112- res.end(JSON.stringify({ type: "message", content: [{ type: "text", text: "fake model says hi" }] }));
112+ res.end(JSON.stringify({ type: "message", content: [{ type: "text", text: "fake model says hi" }], usage: { input_tokens: 21, output_tokens: 4, cache_read_input_tokens: 300 } }));
113113 });
114114 });
115115 await new Promise((r) => upstream.listen(0, "127.0.0.1", r));
@@ -138,7 +138,8 @@
138138 // A repository can set config that runs commands; in the task's clone it only ever runs in a jail.
139139 "git config core.fsmonitor 'touch ../pwned-fsmonitor; false'",
140140 'echo "made in the terminal" > terminal.txt',
141- 'echo \'{"type":"result","subtype":"success","result":"Done in the terminal."}\'',
141+ // What it reports about its own usage: a subscription's requests never pass Diffinery (phase 5a).
142+ 'echo \'{"type":"result","subtype":"success","result":"Done in the terminal.","total_cost_usd":0.0312,"usage":{"input_tokens":40,"output_tokens":7,"cache_read_input_tokens":1200,"cache_creation_input_tokens":90}}\'',
142143 ].join("\n"),
143144 { mode: 0o755 },
144145 );
modifiedapps/diffinery/test/e2e/tasks.test.mjs+29−0View file
@@ -182,6 +182,35 @@
182182 assert.match(taskPage, /Add a hello file/);
183183 assert.match(taskPage, /fake model says hi/);
184184
185+ // Usage (phase 5a): counted by the model proxy from the provider's answer, on the task, its page and the person's 30 days.
186+ const counted = await until("the run's usage", async () => {
187+ const t = await ok("GET", `${R}/${n}`);
188+ return t.runs[0].usage?.input_tokens === 21 ? t : null;
189+ });
190+ assert.deepEqual(counted.runs[0].usage, { source: "proxy", requests: 1, input_tokens: 21, output_tokens: 4, cache_read_tokens: 300, cache_write_tokens: 0, cost_usd: null });
191+ assert.deepEqual(counted.usage, { requests: 1, input_tokens: 21, output_tokens: 4, cache_read_tokens: 300, cache_write_tokens: 0, cost_usd: null });
192+ assert.equal((await mcp("agent_task", { owner, repo: "tasks-demo", number: n })).structuredContent.usage.input_tokens, 21);
193+ const usagePage = await (await fetch(`${base}/${owner}/tasks-demo/tasks/${n}`)).text();
194+ assert.match(usagePage, /<h2>Usage<\/h2>/);
195+ assert.match(usagePage, /Diffinery<\/td>/);
196+ const report = await until("the usage report", async () => {
197+ const r = await ok("GET", "/diffinery/model-usage");
198+ return r.repositories.find((x) => x.repository === `${owner}/tasks-demo` && x.kind === "task" && x.input_tokens >= 21) ? r : null;
199+ });
200+ assert.equal(report.days, 30);
201+ assert.deepEqual(report.repositories[0], { repository: `${owner}/tasks-demo`, kind: "task", runs: 1, reported_by_cli_runs: 0, requests: 1, input_tokens: 21, output_tokens: 4, cache_read_tokens: 300, cache_write_tokens: 0, cost_usd: null });
202+ assert.equal((await ok("GET", "/diffinery/model-usage?days=500")).days, 90);
203+ // A repository is named as it was when the run ended, not by a later name (Security, 5a).
204+ await ok("PATCH", `/repos/${owner}/tasks-demo`, { name: "tasks-demo-renamed" });
205+ assert.deepEqual((await ok("GET", "/diffinery/model-usage")).repositories.map((x) => x.repository), [`${owner}/tasks-demo`]);
206+ await ok("PATCH", `/repos/${owner}/tasks-demo-renamed`, { name: "tasks-demo" });
207+ const settingsPage = await (await fetch(`${base}/settings/model-access`)).text();
208+ assert.match(settingsPage, /Usage in the last 30 days/);
209+ assert.ok(settingsPage.includes(`>${owner}/tasks-demo</a>`));
210+ // Someone else's report is their own: the rival pays for nothing here.
211+ const rivalUsage = await (await fetch(`${base}/api/v3/diffinery/model-usage`, { headers: { cookie: `__Host-dfy_session=${RIVAL_SESSION}` } })).json();
212+ assert.deepEqual(rivalUsage.repositories, []);
213+
185214 // Follow-ups and retries: the person, in a browser; never a token.
186215 assert.equal((await api("POST", `${R}/${n}/messages`, { body: "again" })).status, 403);
187216 assert.equal((await api("POST", `${R}/${n}/retry`, {})).status, 403);
modifiedapps/diffinery/test/e2e/terminals.test.mjs+10−0View file
@@ -305,6 +305,16 @@
305305 assert.equal(twice.runs[1].state, "done");
306306 assert.equal(twice.runs[1].in_terminal, true);
307307 assert.equal(twice.runs[1].kind, "message");
308+ // Usage (phase 5a): no request passed Diffinery, so each run shows what the signed-in CLI reported, with its cost.
309+ assert.deepEqual(twice.runs[0].usage, { source: "runtime", requests: 0, input_tokens: 40, output_tokens: 7, cache_read_tokens: 1200, cache_write_tokens: 90, cost_usd: 0.0312 });
310+ assert.deepEqual(twice.usage, { requests: 0, input_tokens: 80, output_tokens: 14, cache_read_tokens: 2400, cache_write_tokens: 180, cost_usd: 0.0624 });
311+ assert.match((await page(`tasks/${n}`)).text, /Claude Code in the terminal/);
312+ const usage = await until("the terminal runs in the usage report", async () => {
313+ const r = await ok("GET", "/diffinery/model-usage");
314+ return r.repositories.find((x) => x.repository === `${owner}/term-tasks` && x.runs === 2) ?? null;
315+ });
316+ assert.equal(usage.reported_by_cli_runs, 2);
317+ assert.equal(usage.cost_usd, 0.0624);
308318
309319 // Ending the terminal ends what it can run: the page no longer offers it, and a click is refused.
310320 assert.equal((await form(`terminals/${id}/end`, {})).status, 303);
modifiedapps/diffinery/test/unit/rate-limit.test.ts+23−1View file
@@ -2,7 +2,7 @@
22 import { describe, expect, it } from "vitest";
33 import type { Env } from "../../src/env.js";
44 import type { Principal } from "../../src/auth/principal.js";
5-import { bucketsFor, overLimit, RATE_LIMITS, RATE_PERIOD_S, type RateLimiter } from "../../src/lib/rate-limit.js";
5+import { bucketsFor, MODEL_RATE_LIMIT, modelRateLimited, overLimit, RATE_LIMITS, RATE_PERIOD_S, type RateLimiter } from "../../src/lib/rate-limit.js";
66
77 const anon: Principal = { type: "anonymous" };
88 const person = { type: "account", account: { id: "u1" }, via: "token", scopes: ["write"], repoIds: null } as unknown as Principal;
@@ -112,3 +112,25 @@
112112 expect(((await r.json()) as { message: string }).message).toMatch(/at most 300 a minute/);
113113 });
114114 });
115+
116+describe("model requests per paying person (phase 5a)", () => {
117+ it("counts per person, and lets requests through without a binding or when it fails", async () => {
118+ const rl = limiter(2);
119+ const env = { RL_MODEL: rl } as unknown as Env;
120+ expect(await modelRateLimited(env, "p1")).toBe(false);
121+ expect(await modelRateLimited(env, "p1")).toBe(false);
122+ expect(await modelRateLimited(env, "p1")).toBe(true);
123+ expect(await modelRateLimited(env, "p2")).toBe(false);
124+ expect(rl.seen).toEqual(["model:p1", "model:p1", "model:p1", "model:p2"]);
125+ expect(await modelRateLimited({} as Env, "p1")).toBe(false);
126+ const broken = { RL_MODEL: { limit: async () => { throw new Error("down"); } } } as unknown as Env;
127+ expect(await modelRateLimited(broken, "p1")).toBe(false);
128+ });
129+
130+ it("matches wrangler.jsonc, and the model proxy is not in the per-account buckets", () => {
131+ const text = readFileSync(new URL("../../wrangler.jsonc", import.meta.url), "utf8");
132+ const line = text.split("\n").find((l) => l.includes('"name": "RL_MODEL"')) ?? "";
133+ expect(/"limit": (\d+), "period": (\d+)/.exec(line)?.slice(1).map(Number)).toEqual([MODEL_RATE_LIMIT, RATE_PERIOD_S]);
134+ expect(bucketsFor("POST", "/-/model/anthropic/v1/messages", person, "203.0.113.9")).toEqual([]);
135+ });
136+});
modifiedapps/diffinery/test/unit/usage.test.ts+43−1View file
@@ -1,5 +1,5 @@
11 import { describe, expect, it } from "vitest";
2-import { COMPACT_MAX_OUTPUT, DEFAULT_MAX_OUTPUT, formatTokens, pinRefusal, prepareRequest, readBody, readUsage, usageOf, weighted } from "../../src/agentic/usage.js";
2+import { COMPACT_MAX_OUTPUT, DEFAULT_MAX_OUTPUT, formatCost, formatTokens, pinRefusal, prepareRequest, readBody, readUsage, reportedUsage, runUsage, totalUsage, usageJson, usageOf, weighted } from "../../src/agentic/usage.js";
33
44 const stream = (chunks: string[]) =>
55 new ReadableStream<Uint8Array>({
@@ -161,3 +161,45 @@
161161 expect(await readBody(null, 5)).toEqual(new Uint8Array());
162162 });
163163 });
164+
165+describe("usage a CLI reports (phase 5a)", () => {
166+ it("reads Claude Code's result totals and cost, and Codex's per-turn tokens", () => {
167+ const result = JSON.stringify({ type: "result", subtype: "success", result: "x".repeat(50_000), total_cost_usd: 0.4123, usage: { input_tokens: 12, output_tokens: 340, cache_read_input_tokens: 9000, cache_creation_input_tokens: 800 } });
168+ expect(reportedUsage("claude-code", result)).toEqual({ usage: { input: 12, output: 340, cacheRead: 9000, cacheWrite: 800 }, costUsd: 0.4123, mode: "set" });
169+ const turn = JSON.stringify({ type: "turn.completed", usage: { input_tokens: 1200, cached_input_tokens: 1000, output_tokens: 55 } });
170+ expect(reportedUsage("codex", turn)).toEqual({ usage: { input: 200, output: 55, cacheRead: 1000, cacheWrite: 0 }, costUsd: null, mode: "add" });
171+ });
172+
173+ it("reads nothing from other runtimes, other events, or a cost that isn't a number", () => {
174+ const result = JSON.stringify({ type: "result", total_cost_usd: 1, usage: { input_tokens: 1 } });
175+ expect(reportedUsage("custom", result)).toBeNull();
176+ expect(reportedUsage("codex", result)).toBeNull();
177+ expect(reportedUsage("claude-code", JSON.stringify({ type: "assistant", usage: { input_tokens: 1 } }))).toBeNull();
178+ expect(reportedUsage("claude-code", '{"type":"result", cut')).toBeNull();
179+ expect(reportedUsage("claude-code", JSON.stringify({ type: "result", result: "no usage" }))).toBeNull();
180+ expect(reportedUsage("claude-code", JSON.stringify({ type: "result", total_cost_usd: -2, usage: { input_tokens: 1 } }))!.costUsd).toBeNull();
181+ expect(reportedUsage("claude-code", JSON.stringify({ type: "result", total_cost_usd: "9", usage: { input_tokens: 1 } }))!.costUsd).toBeNull();
182+ });
183+
184+ const row = { model_calls: 0, input_tokens: 0, output_tokens: 0, cache_read_tokens: 0, cache_write_tokens: 0, reported_input: 0, reported_output: 0, reported_cache_read: 0, reported_cache_write: 0, cost_usd: null as number | null };
185+
186+ it("takes the proxy's counts when the run used it, else what the CLI reported, else none", () => {
187+ expect(runUsage(row)).toBeNull();
188+ // A run through the proxy keeps Diffinery's counts, even if its CLI also reported some.
189+ expect(runUsage({ ...row, model_calls: 3, input_tokens: 10, output_tokens: 5, reported_input: 999, cost_usd: 0.02 })).toEqual({ source: "proxy", requests: 3, input: 10, output: 5, cacheRead: 0, cacheWrite: 0, costUsd: 0.02 });
190+ expect(runUsage({ ...row, reported_input: 7, reported_output: 2, reported_cache_read: 100 })).toEqual({ source: "runtime", requests: 0, input: 7, output: 2, cacheRead: 100, cacheWrite: 0, costUsd: null });
191+ expect(runUsage({ ...row, cost_usd: 0 })).toEqual({ source: "runtime", requests: 0, input: 0, output: 0, cacheRead: 0, cacheWrite: 0, costUsd: 0 });
192+ });
193+
194+ it("adds runs up, with a cost only from the runs that reported one", () => {
195+ const a = runUsage({ ...row, model_calls: 2, input_tokens: 10, output_tokens: 1, cost_usd: 0.5 });
196+ const b = runUsage({ ...row, reported_input: 5, reported_output: 4 });
197+ expect(totalUsage([a, null, b])).toEqual({ requests: 2, input: 15, output: 5, cacheRead: 0, cacheWrite: 0, costUsd: 0.5 });
198+ expect(totalUsage([b])!.costUsd).toBeNull();
199+ expect(totalUsage([null])).toBeNull();
200+ expect(usageJson({ ...a!, costUsd: 0.123456 })).toEqual({ source: "proxy", requests: 2, input_tokens: 10, output_tokens: 1, cache_read_tokens: 0, cache_write_tokens: 0, cost_usd: 0.1235 });
201+ expect(formatCost(0.004)).toBe("<$0.01");
202+ expect(formatCost(0)).toBe("$0.00");
203+ expect(formatCost(1.416)).toBe("$1.42");
204+ });
205+});
modifiedapps/diffinery/wrangler.jsonc+3−1View file
@@ -48,7 +48,9 @@
4848 { "name": "RL_AUTH", "namespace_id": "1001", "simple": { "limit": 20, "period": 60 } },
4949 { "name": "RL_ANON", "namespace_id": "1002", "simple": { "limit": 300, "period": 60 } },
5050 { "name": "RL_ACCOUNT", "namespace_id": "1003", "simple": { "limit": 2000, "period": 60 } },
51- { "name": "RL_CREATE", "namespace_id": "1004", "simple": { "limit": 60, "period": 60 } }
51+ { "name": "RL_CREATE", "namespace_id": "1004", "simple": { "limit": 60, "period": 60 } },
52+ // Model requests through the proxy (/-/model/*), per paying person (agentic/proxy.ts MODEL_RATE_LIMIT).
53+ { "name": "RL_MODEL", "namespace_id": "1005", "simple": { "limit": 600, "period": 60 } }
5254 ],
5355 "durable_objects": {
5456 "bindings": [