Diffinery

codician/diffinerypublic

Open-source, agent-first GitHub and GitHub Actions alternative that runs on your own Cloudflare account: git hosting, pull requests, code review and drop-in GitHub Actions YAML on Workers, Durable Objects, R2 and Containers.

Agent review: a token budget per review, booked before each model request (#71)

* Agent review: a token budget per review, and autofix counts toward the daily limit

Security's follow-ups to #70:

- The model proxy reads the usage each answer reports (Anthropic Messages,
  OpenAI Responses and Chat Completions, streamed or not) from a copy of the
  response and charges the run. A review stops once it reaches its token
  budget (default 2M, 100k to 50M, cache reads counting as a tenth), set
  next to the daily limit as tokens_per_review. Every run shows its model
  requests and tokens, in the transcript and the API.
- Autofix tasks' runs count toward the repository's daily limit, and autofix
  doesn't start once it is reached.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014VDQqT8oL18uLJ9fpsNxW2

* Model proxy: charge every answer, even one without usage

Security's change on #71: an answer that reports no usage was free, which a
steered agent could arrange with streamed Chat Completions without
include_usage, or a background response it never fetches.

- An answer to a generating request (messages, responses, chat completions)
  that reports no usage is charged an estimate: a token per 3 bytes of the
  request plus the output it allows (max_tokens and the like, or 32k).
- Streamed Chat Completions are asked to include usage.
- A review can't start a background response.
- An SSE line over 1 MB is dropped rather than buffered.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014VDQqT8oL18uLJ9fpsNxW2

* Model proxy: book each request's estimate before it goes upstream

Requests sent at once could each pass the budget check and together overrun
a review's token budget, since usage was charged only after each response.
The proxy now reads and prepares the request first, and modelAuth books its
estimate (the most it may cost) against the run before it is forwarded,
refusing it when that could pass the budget. chargeUsage settles the booking
to the reported usage; a response whose usage can't be read keeps its
booking, and an error response or a request that never reaches the provider
is refunded.

Reviews also can't use previous_response_id, conversation or prompt, whose
stored context the estimate can't see, and the request body is read with a
running byte count.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014VDQqT8oL18uLJ9fpsNxW2

* Model proxy: reviews can't ask for n answers, and uncapped requests are capped

A booking counts one answer of the request's output cap, so a review may not
ask for more than one answer (n), and a review's request that sets no output
cap is sent with the cap its estimate assumed (32,000 tokens).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014VDQqT8oL18uLJ9fpsNxW2

* Agent review settings: describe the token budget as the proxy now enforces it

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014VDQqT8oL18uLJ9fpsNxW2

---------

Co-authored-by: Claude <noreply@anthropic.com>
Canberk Demirci authored · GitHub committed signed1 parent 2805acbcommit d59844f0a865c0bff7071f380657b522c5f087b0Browse files
13 files changed+569−39
Files (13)
modifiedapps/diffinery/docs/agents/review.md+15−4View file
@@ -104,15 +104,26 @@
104104 | --- | --- |
105105 | **30 minutes** per review | The review stops, and its check fails. |
106106 | **200 model requests** per review | The model proxy refuses the next request and stops the review, and its check fails. |
107+| **Tokens per review** (default 2,000,000, from 100,000 to 50,000,000) | The same, once the next request could take the review past its budget. |
107108 | **10 automatic reviews** per pull request in 24 hours | Later pushes get an action-required "Waiting for a request" check. People with write access can still ask. |
108109 | **20 reviews** per pull request in 24 hours, asked for or automatic | The check asks for action, and asking returns `429`. |
109-| The repository's **daily limit** (default 50, from 1 to 1000) | The check asks for action, asking returns `429`, and the admin whose key reviews use gets an email, at most once a day. |
110+| The repository's **daily limit** (default 50, from 1 to 1000) | The check asks for action, asking returns `429`, autofix waits, and the admin whose key reviews use gets an email, at most once a day. |
110111
111-Daily limits count the reviews started in the last 24 hours, so reviews run again as older ones fall out of that window. Change the repository's limit under **Settings → Agent review**.
112+Daily limits count the reviews and autofix runs started in the last 24 hours, so reviews run again as older ones fall out of that window.
112113
114+Diffinery counts tokens itself, as requests pass through the model proxy:
115+
116+- Before a request goes to the provider, the proxy books the most it may cost: a token per 3 bytes of request, plus the output it allows (`max_tokens`, or 32,000 when it sets none). If that could take the review past its budget, the request is refused and the review stops. So requests sent at once can't overrun the budget together.
117+- Once the response has streamed through, the booking is replaced by the usage the provider reports: input, output and cache writes in full, cache reads as a tenth, since providers bill them at about a tenth. A response that reports no usage, or whose usage can't be read, stays charged its booking. An error response costs nothing.
118+- A review can't start background responses, use `previous_response_id`, `conversation` or `prompt` (they pull in context stored at the provider that the booking can't see), or ask for more than one answer (`n`). A review's request that sets no output cap is sent with one of 32,000 tokens, so its booking holds.
119+
120+Every run's model requests and tokens show in its transcript and in the API's `runs` (`model_requests`, `tokens`).
121+
122+Change the daily limit and the token budget under **Settings → Agent review**.
123+
113124 ## Who pays, and the risks
114125
115-- Every review spends the model key of the admin who turned reviews on. Use a **dedicated key with a spending limit** with your provider as well: Diffinery's [limits](#limits) count reviews and model requests, not tokens or money.
126+- Every review spends the model key of the admin who turned reviews on. Use a **dedicated key with a spending limit** with your provider as well: Diffinery's [limits](#limits) count reviews, requests and tokens, not money.
116127 - While a review runs, anything in its sandbox, including code from the pull request, can use that key through the model proxy. The proxy forwards only the model endpoints, and the review's model token stops working when the agent exits (see [How your key is protected](/docs/agents/tasks#how-your-key-is-protected)).
117128 - The sandbox can't push or write repository content. Its credential acts as the agent with read access.
118129
@@ -121,7 +132,7 @@
121132 | Endpoint | Does | Who |
122133 | --- | --- | --- |
123134 | `GET /repos/{owner}/{repo}/diffinery/agent-review` | The repository's agent review settings | Write access |
124-| `PUT /repos/{owner}/{repo}/diffinery/agent-review` | Turn on, change (`agent`, `runtime`, `command`, `trigger`: `push`, `open` or `request`, `authors`: `all`, `people` or `agents`, `drafts`, `autofix`, `daily_limit`: 1 to 1000, `accept_terms: true`) or turn off (`enabled: false`) | Repository admin, browser session only |
135+| `PUT /repos/{owner}/{repo}/diffinery/agent-review` | Turn on, change (`agent`, `runtime`, `command`, `trigger`: `push`, `open` or `request`, `authors`: `all`, `people` or `agents`, `drafts`, `autofix`, `daily_limit`: 1 to 1000, `tokens_per_review`, `accept_terms: true`) or turn off (`enabled: false`) | Repository admin, browser session only |
125136 | `GET /repos/{owner}/{repo}/pulls/{number}/diffinery/agent-review` | The pull request's review task, the last reviewed head and every finding with its state (`open`, `fixed`, `dismissed`) | Write access |
126137 | `POST /repos/{owner}/{repo}/pulls/{number}/diffinery/agent-review` | Ask for a review (`429` past a daily limit) | People with write access (not agents) |
127138
modifiedapps/diffinery/docs/agents/tasks.md+3−1View file
@@ -149,10 +149,12 @@
149149 "queued_message": null,
150150 "error": null,
151151 "html_url": "https://diffinery.com/acme/website/tasks/7",
152- "runs": [{ "number": 1, "kind": "start", "message": null, "state": "done", "exit_code": 0, "error": null, "started_at": "2026-10-14T07:02:11.000Z", "ended_at": "2026-10-14T07:09:40.000Z" }]
152+ "runs": [{ "number": 1, "kind": "start", "message": null, "state": "done", "exit_code": 0, "error": null, "model_requests": 42, "tokens": 318204, "started_at": "2026-10-14T07:02:11.000Z", "ended_at": "2026-10-14T07:09:40.000Z" }]
153153 }
154154 ```
155155
156+A run's `model_requests` and `tokens` are what its agent used through the model proxy, counted from the usage the provider reports (cache reads count as a tenth). An answer that reports no usage counts an estimate: a token per 3 bytes of request, plus the output it allows.
157+
156158 Transcript lines have `seq`, `run`, `at`, `stream` (`out` and `err` for the runtime's output, `setup` for installing and pushing, `note` for Diffinery's own lines) and `text`. Claude Code writes `stream-json` lines and Codex writes `--json` lines to `out`.
157159
158160 ## MCP
modifiedapps/diffinery/src/agentic/ProjectDO.ts+79−15View file
@@ -4,9 +4,10 @@
44 import { HttpError, now, randomToken, sha256 } from "../lib/util.js";
55 import { atLeast } from "../repo/types.js";
66 import { taskCaller, TaskAccessError } from "./access.js";
7-import type { Provider } from "./keys.js";
7+import { PROVIDERS, type Provider } from "./keys.js";
88 import { enqueueMail } from "../mail/send.js";
9-import { countText, dayAgo, findingComment, rulesFiles, findingId, MAX_AUTOFIXES, placeFindings, REPO_DAILY_DEFAULT, REVIEW_APP, REVIEW_CHECK, REVIEW_CREATOR, REVIEW_MAX_MODEL_CALLS, REVIEW_TIMEOUT_MS, reviewBody, reviewLimit, reviewOutput, reviewPrompt, fixGoal, shouldMailLimit, type Finding, type PriorFinding, type ReviewLimit, type ReviewSettings } from "./review.js";
9+import { formatTokens, weighted, type Usage } from "./usage.js";
10+import { countText, dayAgo, findingComment, rulesFiles, findingId, MAX_AUTOFIXES, placeFindings, REPO_DAILY_DEFAULT, REVIEW_APP, REVIEW_CHECK, REVIEW_CREATOR, REVIEW_MAX_MODEL_CALLS, REVIEW_TIMEOUT_MS, REVIEW_TOKENS_DEFAULT, reviewBody, reviewLimit, reviewOutput, reviewPrompt, fixGoal, shouldMailLimit, type Finding, type PriorFinding, type ReviewLimit, type ReviewSettings } from "./review.js";
1011 import { agentEnv, finalMessage, INSTALL_TIMEOUT_MS, RUN_TIMEOUT_MS, RUNTIMES, runPrompt, taskBranch, taskTitle, type RuntimeId, type TranscriptLine } from "./runtimes.js";
1112
1213 /**
@@ -54,6 +55,8 @@
5455 reviewed_sha: string | null;
5556 /** Reviews: fix tasks started by autofix for this pull request. */
5657 autofixes: number;
58+ /** An autofix task: the review task that started it (its runs count toward the daily review limit). */
59+ autofix_of: number | null;
5760 created_at: string;
5861 updated_at: string;
5962 };
@@ -77,6 +80,12 @@
7780 token_hash: string | null;
7881 /** Requests the run's agent made through the model proxy. */
7982 model_calls: number;
83+ /** Tokens the provider reported for those requests (usage.ts), and the weighted total a budget counts. */
84+ tokens: number;
85+ input_tokens: number;
86+ output_tokens: number;
87+ cache_read_tokens: number;
88+ cache_write_tokens: number;
8089 exit_code: number | null;
8190 error: string | null;
8291 checked_at: string | null;
@@ -85,7 +94,7 @@
8594 };
8695
8796 export interface TaskView extends TaskRow {
88- runs: Array<Pick<RunRow, "seq" | "kind" | "message" | "step" | "exit_code" | "error" | "started_at" | "ended_at">>;
97+ runs: Array<Pick<RunRow, "seq" | "kind" | "message" | "step" | "exit_code" | "error" | "model_calls" | "tokens" | "started_at" | "ended_at">>;
8998 }
9099
91100 const SCHEMA = `
@@ -173,6 +182,12 @@
173182 ["tasks", "autofixes", "autofixes INTEGER NOT NULL DEFAULT 0"],
174183 ["runs", "head_sha", "head_sha TEXT"],
175184 ["runs", "model_calls", "model_calls INTEGER NOT NULL DEFAULT 0"],
185+ ["runs", "tokens", "tokens INTEGER NOT NULL DEFAULT 0"],
186+ ["runs", "input_tokens", "input_tokens INTEGER NOT NULL DEFAULT 0"],
187+ ["runs", "output_tokens", "output_tokens INTEGER NOT NULL DEFAULT 0"],
188+ ["runs", "cache_read_tokens", "cache_read_tokens INTEGER NOT NULL DEFAULT 0"],
189+ ["runs", "cache_write_tokens", "cache_write_tokens INTEGER NOT NULL DEFAULT 0"],
190+ ["tasks", "autofix_of", "autofix_of INTEGER"],
176191 ];
177192
178193 /** A finding as stored. */
@@ -216,6 +231,8 @@
216231 providers: Provider[];
217232 /** Work on an existing pull request's branch (Fix it, autofix) instead of a new branch. */
218233 pull?: { number: number; head: string };
234+ /** Autofix: the review task that started it. */
235+ autofixOf?: number;
219236 }
220237
221238 export interface ReviewRequest {
@@ -255,7 +272,7 @@
255272 get(number: number): TaskView | null {
256273 const t = this.task(number);
257274 if (!t) return null;
258- const runs = this.sql.exec<RunRow>("SELECT seq, kind, message, step, exit_code, error, started_at, ended_at FROM runs WHERE task = ? ORDER BY seq", number).toArray();
275+ const runs = this.sql.exec<RunRow>("SELECT seq, kind, message, step, exit_code, error, model_calls, tokens, started_at, ended_at FROM runs WHERE task = ? ORDER BY seq", number).toArray();
259276 return { ...t, runs };
260277 }
261278
@@ -270,7 +287,7 @@
270287 const at = now();
271288 const branch = input.pull?.head ?? taskBranch(number, input.goal);
272289 this.sql.exec(
273- "INSERT INTO tasks (number, id, repo_id, goal, title, runtime, command, agent_id, agent_login, person_id, person_login, base, branch, state, pr_number, created_at, updated_at) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 'running', ?, ?, ?)",
290+ "INSERT INTO tasks (number, id, repo_id, goal, title, runtime, command, agent_id, agent_login, person_id, person_login, base, branch, state, pr_number, autofix_of, created_at, updated_at) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 'running', ?, ?, ?, ?)",
274291 number,
275292 crypto.randomUUID(),
276293 input.repoId,
@@ -285,6 +302,7 @@
285302 input.base,
286303 branch,
287304 input.pull?.number ?? null,
305+ input.autofixOf ?? null,
288306 at,
289307 at,
290308 );
@@ -415,6 +433,11 @@
415433 return { task: t.number, status: "started" };
416434 }
417435
436+ /** Runs the repository's daily review limit counts: reviews and autofix tasks' runs started in the last 24 hours. */
437+ private dailyRuns(): number {
438+ return this.sql.exec<{ n: number }>("SELECT count(*) AS n FROM runs r JOIN tasks t ON t.number = r.task WHERE r.started_at > ? AND (t.kind = 'review' OR t.autofix_of IS NOT NULL)", dayAgo()).one().n;
439+ }
440+
418441 /**
419442 * The daily limits: when one is reached, the head gets an action-required
420443 * check instead of a review, and the payer an email (once a day) when it is
@@ -423,9 +446,8 @@
423446 private async overLimit(t: TaskRow, headSha: string, automatic: boolean): Promise<ReviewLimit | null> {
424447 const settings = this.reviewSettings();
425448 if (!settings) return null;
426- const since = dayAgo();
427- const count = (where: string, ...args: unknown[]) => this.sql.exec<{ n: number }>(`SELECT count(*) AS n FROM runs WHERE kind = 'review' AND started_at > ? ${where}`, since, ...args).one().n;
428- const limit = reviewLimit({ repo: count(""), pr: count("AND task = ?", t.number), prAutomatic: count("AND task = ? AND message IS NULL", t.number) }, automatic, settings);
449+ const count = (where: string, ...args: unknown[]) => this.sql.exec<{ n: number }>(`SELECT count(*) AS n FROM runs WHERE kind = 'review' AND started_at > ? ${where}`, dayAgo(), ...args).one().n;
450+ const limit = reviewLimit({ repo: this.dailyRuns(), pr: count("AND task = ?", t.number), prAutomatic: count("AND task = ? AND message IS NULL", t.number) }, automatic, settings);
429451 if (!limit) return null;
430452 await this.reviewCheck(t, headSha, "completed", "action_required", limit.title, limit.summary);
431453 if (limit.scope === "repo" && shouldMailLimit(settings)) {
@@ -436,25 +458,65 @@
436458 return limit;
437459 }
438460
439- /** The model proxy's check: which run a proxy token belongs to, while that run's agent is running. */
440- async modelAuth(token: string): Promise<{ task: number; run: number; personId: string; agentId: string; repoId: string; providers: Provider[] } | { limit: string } | null> {
461+ /**
462+ * The model proxy's check, before each request goes upstream: which run a proxy token belongs to, while that
463+ * run's agent is running, and whether the run may send this request. The request's estimate (the most it may
464+ * cost) is booked here, before it is sent, so requests in flight at once can't together overrun a review's
465+ * budget; chargeUsage settles it to what the answer reports.
466+ */
467+ async modelAuth(
468+ token: string,
469+ req: { provider: Provider; estimate: Usage | null; reviewRefuse?: string },
470+ ): Promise<{ task: number; run: number; personId: string; agentId: string; repoId: string; review: boolean } | { limit: string } | { refuse: string } | null> {
441471 const hash = await sha256(token);
442472 const r = this.sql.exec<RunRow>("SELECT * FROM runs WHERE token_hash = ? AND step = 'agent'", hash).toArray()[0];
443473 if (!r) return null;
444474 const t = this.task(r.task)!;
475+ if (!(JSON.parse(r.providers) as Provider[]).includes(req.provider)) return { refuse: `this task doesn't use ${PROVIDERS[req.provider].name}` };
476+ if (t.kind === "review" && req.reviewRefuse) return { refuse: req.reviewRefuse };
445477 const calls = r.model_calls + 1;
446- this.sql.exec("UPDATE runs SET model_calls = ? WHERE task = ? AND seq = ?", calls, r.task, r.seq);
478+ const cost = req.estimate ? weighted(req.estimate) : 0;
447479 if (t.kind === "review") {
448480 // A review's budget is charged here, not trusted to the agent: its sandbox runs the pull request's code.
449- const limit = Date.parse(r.started_at) + REVIEW_TIMEOUT_MS < Date.now() ? `The review took longer than ${REVIEW_TIMEOUT_MS / 60_000} minutes and was stopped.` : calls > REVIEW_MAX_MODEL_CALLS ? `The review used its limit of ${REVIEW_MAX_MODEL_CALLS} model requests and was stopped.` : null;
481+ const budget = this.reviewSettings()?.tokensPerReview ?? REVIEW_TOKENS_DEFAULT;
482+ const limit =
483+ Date.parse(r.started_at) + REVIEW_TIMEOUT_MS < Date.now()
484+ ? `The review took longer than ${REVIEW_TIMEOUT_MS / 60_000} minutes and was stopped.`
485+ : calls > REVIEW_MAX_MODEL_CALLS
486+ ? `The review used its limit of ${REVIEW_MAX_MODEL_CALLS} model requests and was stopped.`
487+ : r.tokens >= budget
488+ ? `The review used its budget of ${formatTokens(budget)} tokens (${formatTokens(r.tokens)}) and was stopped.`
489+ : r.tokens + cost > budget
490+ ? `The review's next model request could take it past its budget of ${formatTokens(budget)} tokens (${formatTokens(r.tokens)} used), so it was stopped.`
491+ : null;
450492 if (limit) {
493+ this.sql.exec("UPDATE runs SET model_calls = ? WHERE task = ? AND seq = ?", calls, r.task, r.seq);
451494 await this.endRun(r, "failed", limit);
452495 return { limit };
453496 }
454497 }
455- return { task: t.number, run: r.seq, personId: t.person_id, agentId: t.agent_id, repoId: t.repo_id, providers: JSON.parse(r.providers) as Provider[] };
498+ this.sql.exec("UPDATE runs SET model_calls = ?, tokens = tokens + ? WHERE task = ? AND seq = ?", calls, cost, r.task, r.seq);
499+ return { task: t.number, run: r.seq, personId: t.person_id, agentId: t.agent_id, repoId: t.repo_id, review: t.kind === "review" };
456500 }
457501
502+ /**
503+ * Settle a request booked by modelAuth: `used` is what the answer reports (or its estimate, when it reports
504+ * none), and replaces the `booked` estimate. A request whose usage can't be read is never settled, and so
505+ * stays charged its estimate.
506+ */
507+ async chargeUsage(task: number, run: number, used: Usage, booked: Usage | null): Promise<void> {
508+ this.sql.exec(
509+ "UPDATE runs SET tokens = MAX(tokens + ?, 0), input_tokens = input_tokens + ?, output_tokens = output_tokens + ?, cache_read_tokens = cache_read_tokens + ?, cache_write_tokens = cache_write_tokens + ? WHERE task = ? AND seq = ?",
510+ weighted(used) - (booked ? weighted(booked) : 0),
511+ used.input,
512+ used.output,
513+ used.cacheRead,
514+ used.cacheWrite,
515+ task,
516+ run,
517+ );
518+ }
519+
458520 // ---------------------------------------------------------------- runs
459521
460522 private startRun(number: number, kind: RunRow["kind"], message: string | null, providers: Provider[]): void {
@@ -526,7 +588,7 @@
526588 const run = this.run(r.task, r.seq)!;
527589 // The agent is done: its model token stops working now, before anything else runs.
528590 this.sql.exec("UPDATE runs SET exit_code = ?, token_hash = NULL WHERE task = ? AND seq = ?", done, r.task, r.seq);
529- this.note(r.task, r.seq, done === 0 ? `${RUNTIMES[t.runtime].name} finished.` : `${RUNTIMES[t.runtime].name} exited with code ${done}.`);
591+ this.note(r.task, r.seq, `${done === 0 ? `${RUNTIMES[t.runtime].name} finished.` : `${RUNTIMES[t.runtime].name} exited with code ${done}.`}${run.model_calls ? ` ${run.model_calls} model ${run.model_calls === 1 ? "request" : "requests"}, ${formatTokens(run.tokens)} tokens.` : ""}`);
530592 return this.setStep(run, "finish");
531593 }
532594 case "finish": {
@@ -704,10 +766,12 @@
704766 const settings = this.reviewSettings();
705767 const fresh = this.task(t.number)!;
706768 if (settings?.enabled && settings.autofix && blocking.length && (await this.authorMayWrite(t.repo_id, issue.author.id))) {
769+ const repoLimit = settings.dailyLimit ?? REPO_DAILY_DEFAULT;
707770 if (fresh.autofixes >= MAX_AUTOFIXES) this.note(t.number, r.seq, `Autofix has already run ${MAX_AUTOFIXES} times on #${pr}; fix the rest by hand or with Fix it.`);
771+ else if (this.dailyRuns() >= repoLimit) this.note(t.number, r.seq, `Autofix didn't start: the repository reached its daily limit of ${repoLimit} reviews and autofix runs.`);
708772 else {
709773 this.sql.exec("UPDATE tasks SET autofixes = autofixes + 1 WHERE number = ?", t.number);
710- const fix = await this.create({ repoId: t.repo_id, goal: fixGoal(pr, blocking), runtime: settings.runtime, command: settings.command, agent: settings.agent, person: settings.person, base: issue.pull.base, providers: JSON.parse(r.providers) as Provider[], pull: { number: pr, head: issue.pull.head } });
774+ const fix = await this.create({ repoId: t.repo_id, goal: fixGoal(pr, blocking), runtime: settings.runtime, command: settings.command, agent: settings.agent, person: settings.person, base: issue.pull.base, providers: JSON.parse(r.providers) as Provider[], pull: { number: pr, head: issue.pull.head }, autofixOf: t.number });
711775 this.note(t.number, r.seq, `Autofix started task #${fix.number} for the blocking ${blocking.length === 1 ? "finding" : "findings"}.`);
712776 }
713777 }
modifiedapps/diffinery/src/agentic/api.ts+1−1View file
@@ -130,7 +130,7 @@
130130 const b = await jsonBody<SettingsInput & { accept_terms?: unknown }>(c);
131131 const s = await saveReviewSettings(c, l, { ...b, accept: b.accept_terms });
132132 const me = requireAccount(c);
133- c.executionCtx.waitUntil(c.get("dir").audit({ actorId: me.id, actorLogin: me.login, action: "review.settings", target: full(l), detail: { enabled: s?.enabled ?? false, agent: s?.agent.login, runtime: s?.runtime, trigger: s?.trigger, autofix: s?.autofix, daily_limit: s?.dailyLimit } }));
133+ c.executionCtx.waitUntil(c.get("dir").audit({ actorId: me.id, actorLogin: me.login, action: "review.settings", target: full(l), detail: { enabled: s?.enabled ?? false, agent: s?.agent.login, runtime: s?.runtime, trigger: s?.trigger, autofix: s?.autofix, daily_limit: s?.dailyLimit, tokens_per_review: s?.tokensPerReview } }));
134134 return c.json(reviewSettingsJson(s));
135135 });
136136
modifiedapps/diffinery/src/agentic/proxy.ts+42−5View file
@@ -2,7 +2,11 @@
22 import { Directory } from "../lib/directory.js";
33 import { taskPersonAllowed } from "./access.js";
44 import { isProvider, ModelKeys, PROVIDERS, type Provider } from "./keys.js";
5+import { NO_USAGE, prepareRequest, readBody, readUsage } from "./usage.js";
56
7+/** The largest model request the proxy forwards (it reads each one to charge it). */
8+const MAX_REQUEST = 32 * 1024 * 1024;
9+
610 /**
711 * The model proxy: `/-/model/<provider>/v1/...`. A task's runtime calls it
812 * with the run's token (`dmt_...`, as its API key) and the proxy forwards the
@@ -63,26 +67,59 @@
6367 if (!repoId || !/^[A-Za-z0-9-]{8,64}$/.test(repoId)) return err(401, "malformed model token");
6468 const env = c.env;
6569 const project = env.PROJECT.get(env.PROJECT.idFromName(repoId));
66- const run = await project.modelAuth(token);
70+ // Read and prepare the request first: its estimate is booked against the run before it goes upstream.
71+ let body: string | undefined;
72+ let prepared: ReturnType<typeof prepareRequest> = { body: "", estimate: null };
73+ if (!["GET", "HEAD"].includes(c.req.method)) {
74+ if (Number(c.req.header("content-length") ?? 0) > MAX_REQUEST) return err(413, "the request is too large");
75+ const bytes = await readBody(c.req.raw.body, MAX_REQUEST);
76+ if (!bytes) return err(413, "the request is too large");
77+ prepared = prepareRequest(c.req.method, rest, new TextDecoder().decode(bytes), bytes.byteLength);
78+ body = prepared.body;
79+ }
80+ const { estimate } = prepared;
81+ const run = await project.modelAuth(token, { provider, estimate, reviewRefuse: prepared.reviewRefuse });
6782 if (!run) return err(401, "this model token isn't valid: its task run has ended");
6883 if ("limit" in run) return err(429, run.limit);
69- if (!run.providers.includes(provider)) return err(403, `this task doesn't use ${PROVIDERS[provider].name}`);
84+ if ("refuse" in run) return err(403, run.refuse);
85+ if (run.review && prepared.reviewBody) body = prepared.reviewBody;
86+ // A booked request that never reaches the provider, or that it answers with an error, costs nothing.
87+ const refund = () => (estimate ? c.executionCtx.waitUntil(project.chargeUsage(run.task, run.run, NO_USAGE, estimate).catch((e) => console.error("refunding a model request failed", e))) : undefined);
7088 // The person must still be allowed to run the task, on every request; otherwise the task stops.
7189 const dir = new Directory(env);
7290 const [repo, agent] = await Promise.all([dir.repoById(run.repoId), dir.account(run.agentId)]);
7391 if (!repo || !agent || !(await taskPersonAllowed(dir, run.personId, agent, repo))) {
92+ refund();
7493 c.executionCtx.waitUntil(project.stop(run.task, "Diffinery", "Stopped: the person who started the task can no longer run it.").then(() => undefined, () => undefined));
7594 return err(403, "the person who started this task can no longer run it");
7695 }
7796 const key = await new ModelKeys(env).reveal(run.personId, provider);
78- if (!key) return err(403, `no ${PROVIDERS[provider].name} key in Settings → Model access`);
97+ if (!key) {
98+ refund();
99+ return err(403, `no ${PROVIDERS[provider].name} key in Settings → Model access`);
100+ }
79101 const headers = new Headers();
80102 for (const [k, v] of c.req.raw.headers) if (!DROP.has(k.toLowerCase())) headers.set(k, v);
81103 if (provider === "anthropic") headers.set("x-api-key", key);
82104 else headers.set("authorization", `Bearer ${key}`);
83105 const base = (provider === "anthropic" ? env.MODEL_UPSTREAM_ANTHROPIC : env.MODEL_UPSTREAM_OPENAI) || UPSTREAM[provider];
84- const res = await fetch(`${base}${rest}${url.search}`, { method: c.req.method, headers, body: ["GET", "HEAD"].includes(c.req.method) ? undefined : c.req.raw.body, redirect: "manual" });
106+ // If the provider can't be reached, the booking stays: it may have started the answer.
107+ const res = await fetch(`${base}${rest}${url.search}`, { method: c.req.method, headers, body, redirect: "manual" });
85108 const out = new Headers(res.headers);
86109 out.delete("set-cookie");
87- return new Response(res.body, { status: res.status, headers: out });
110+ if (!res.body || !res.ok) {
111+ refund();
112+ return new Response(res.body, { status: res.status, headers: out });
113+ }
114+ if (!estimate) return new Response(res.body, { status: res.status, headers: out });
115+ // Settle the booking to the tokens the provider reports, read from a copy of the response as it streams back.
116+ // An answer that reports none is charged its estimate; one whose usage can't be read is never settled, and so
117+ // stays charged its estimate too.
118+ const [agentCopy, meter] = res.body.tee();
119+ c.executionCtx.waitUntil(
120+ readUsage(meter, res.headers.get("content-type") ?? "")
121+ .then((u) => project.chargeUsage(run.task, run.run, u ?? estimate, estimate))
122+ .catch((e) => console.error("reading model usage failed", e)),
123+ );
124+ return new Response(agentCopy, { status: res.status, headers: out });
88125 }
modifiedapps/diffinery/src/agentic/review-web.tsx+9−4View file
@@ -8,7 +8,7 @@
88 import { page as settingsPage } from "../web/repo-settings.js";
99 import { Form, Time, act, post, prm, repoUrl, statusOf, type WebApp } from "../web/ui.js";
1010 import { ModelKeys, PROVIDERS } from "./keys.js";
11-import { AUTHORS, findingId, MAX_AUTOFIXES, PR_DAILY_AUTOMATIC, PR_DAILY_REVIEWS, REPO_DAILY_DEFAULT, REPO_DAILY_MAX, REVIEW_MAX_MODEL_CALLS, REVIEW_TIMEOUT_MS, TRIGGERS, type ReviewSettings } from "./review.js";
11+import { AUTHORS, findingId, MAX_AUTOFIXES, PR_DAILY_AUTOMATIC, PR_DAILY_REVIEWS, REPO_DAILY_DEFAULT, REPO_DAILY_MAX, REVIEW_MAX_MODEL_CALLS, REVIEW_TIMEOUT_MS, REVIEW_TOKENS_DEFAULT, REVIEW_TOKENS_MAX, REVIEW_TOKENS_MIN, TRIGGERS, type ReviewSettings } from "./review.js";
1212 import { askForReview, reviewSettings, runtimeOk, saveReviewSettings } from "./reviews.js";
1313 import { RUNTIMES, type RuntimeId } from "./runtimes.js";
1414 import { drivableAgents } from "./tasks.js";
@@ -131,8 +131,13 @@
131131 <div class="field">
132132 <label for="daily_limit">Daily limit</label>
133133 <input id="daily_limit" name="daily_limit" type="number" min={1} max={REPO_DAILY_MAX} step={1} style="max-width:10em" value={v("daily_limit", String(s.dailyLimit ?? REPO_DAILY_DEFAULT))} />
134+ <div class="hint">The most reviews and autofix runs this repository starts in 24 hours. When it is reached, pull requests get an action-required check instead of a review, autofix waits, and you get an email. Each pull request also gets at most {PR_DAILY_AUTOMATIC} automatic reviews and {PR_DAILY_REVIEWS} in all a day.</div>
135+ </div>
136+ <div class="field">
137+ <label for="tokens_per_review">Tokens per review</label>
138+ <input id="tokens_per_review" name="tokens_per_review" type="number" min={REVIEW_TOKENS_MIN} max={REVIEW_TOKENS_MAX} step={1} style="max-width:12em" value={v("tokens_per_review", String(s.tokensPerReview ?? REVIEW_TOKENS_DEFAULT))} />
134139 <div class="hint">
135- The most reviews this repository runs in 24 hours. When it is reached, pull requests get an action-required check instead of a review, and you get an email. Each pull request also gets at most {PR_DAILY_AUTOMATIC} automatic reviews and {PR_DAILY_REVIEWS} in all a day, and each review stops after {REVIEW_TIMEOUT_MS / 60_000} minutes or {REVIEW_MAX_MODEL_CALLS} model requests.
140+ A review stops when its next model request could take it past this many tokens (cache reads count as a tenth), after {REVIEW_MAX_MODEL_CALLS} model requests, or after {REVIEW_TIMEOUT_MS / 60_000} minutes, whichever comes first. Diffinery books the most each request may cost before sending it, then counts the tokens your provider reports.
136141 </div>
137142 </div>
138143 <div class="flash info">
@@ -166,11 +171,11 @@
166171
167172 post(web, "/:owner/:repo/settings/agent-review", async (c, f) => {
168173 const l = await admin(c);
169- const values = { agent: f.s("agent"), runtime: f.s("runtime") || "claude-code", command: f.s("command"), trigger: f.s("trigger") || "push", authors: f.s("authors") || "all", drafts: f.all("drafts").length ? "1" : "", autofix: f.all("autofix").length ? "1" : "", daily_limit: f.s("daily_limit") };
174+ const values = { agent: f.s("agent"), runtime: f.s("runtime") || "claude-code", command: f.s("command"), trigger: f.s("trigger") || "push", authors: f.s("authors") || "all", drafts: f.all("drafts").length ? "1" : "", autofix: f.all("autofix").length ? "1" : "", daily_limit: f.s("daily_limit"), tokens_per_review: f.s("tokens_per_review") };
170175 try {
171176 const s = await saveReviewSettings(c, l, { ...values, drafts: !!values.drafts, autofix: !!values.autofix, accept: f.all("accept").length > 0 });
172177 const me = requireAccount(c);
173- c.executionCtx.waitUntil(c.get("dir").audit({ actorId: me.id, actorLogin: me.login, action: "review.settings", target: `${l.repo.owner_login}/${l.repo.name}`, detail: { enabled: true, agent: s?.agent.login, runtime: s?.runtime, trigger: s?.trigger, autofix: s?.autofix, daily_limit: s?.dailyLimit } }));
178+ c.executionCtx.waitUntil(c.get("dir").audit({ actorId: me.id, actorLogin: me.login, action: "review.settings", target: `${l.repo.owner_login}/${l.repo.name}`, detail: { enabled: true, agent: s?.agent.login, runtime: s?.runtime, trigger: s?.trigger, autofix: s?.autofix, daily_limit: s?.dailyLimit, tokens_per_review: s?.tokensPerReview } }));
174179 } catch (e) {
175180 const st = statusOf(e);
176181 if (st.status >= 500) throw e;
modifiedapps/diffinery/src/agentic/review.ts+8−2View file
@@ -26,8 +26,10 @@
2626 drafts: boolean;
2727 authors: ReviewAuthors;
2828 autofix: boolean;
29- /** Most reviews the repository runs in 24 hours, set by the payer (REPO_DAILY_DEFAULT when unset). */
29+ /** Most reviews (and autofix runs) the repository runs in 24 hours, set by the payer (REPO_DAILY_DEFAULT when unset). */
3030 dailyLimit?: number;
31+ /** Most tokens one review may use (REVIEW_TOKENS_DEFAULT when unset; see usage.ts for how they count). */
32+ tokensPerReview?: number;
3133 /** When the payer was last emailed that the daily limit was reached. */
3234 limitMailedAt?: string;
3335 optedInAt: string;
@@ -50,6 +52,10 @@
5052 export const REVIEW_TIMEOUT_MS = 30 * 60 * 1000;
5153 /** Model requests (turns) one review may make through the proxy. */
5254 export const REVIEW_MAX_MODEL_CALLS = 200;
55+/** Tokens one review may use, counted from the provider's responses at the proxy, unless the payer sets otherwise. */
56+export const REVIEW_TOKENS_DEFAULT = 2_000_000;
57+export const REVIEW_TOKENS_MIN = 100_000;
58+export const REVIEW_TOKENS_MAX = 50_000_000;
5359 /** Automatic reviews of one pull request in 24 hours; more wait for someone to ask. */
5460 export const PR_DAILY_AUTOMATIC = 10;
5561 /** Reviews of one pull request in 24 hours, asked for or automatic. */
@@ -59,7 +65,7 @@
5965 const DAY_MS = 24 * 60 * 60 * 1000;
6066
6167 export interface ReviewCounts {
62- /** Review runs started in the last 24 hours: in the repository, on this pull request, and automatic ones on it. */
68+ /** Runs started in the last 24 hours: reviews and autofix tasks in the repository, reviews of this pull request, and automatic ones of it. */
6369 repo: number;
6470 pr: number;
6571 prAutomatic: number;
modifiedapps/diffinery/src/agentic/reviews.ts+6−1View file
@@ -8,7 +8,7 @@
88 import { taskCaller, TaskAccessError } from "./access.js";
99 import { ModelKeys, PROVIDERS, type Provider } from "./keys.js";
1010 import type { FindingRow } from "./ProjectDO.js";
11-import { asksForReview, REPO_DAILY_DEFAULT, REPO_DAILY_MAX, REVIEW_APP, REVIEW_CHECK, REVIEW_CREATOR, TRIGGERS, AUTHORS, type ReviewAuthors, type ReviewSettings, type ReviewTrigger } from "./review.js";
11+import { asksForReview, REPO_DAILY_DEFAULT, REPO_DAILY_MAX, REVIEW_TOKENS_DEFAULT, REVIEW_TOKENS_MAX, REVIEW_TOKENS_MIN, REVIEW_APP, REVIEW_CHECK, REVIEW_CREATOR, TRIGGERS, AUTHORS, type ReviewAuthors, type ReviewSettings, type ReviewTrigger } from "./review.js";
1212 import { isRuntime, MAX_COMMAND, RUNTIMES, type RuntimeId } from "./runtimes.js";
1313 import { projectStub, requirePersonSession } from "./tasks.js";
1414
@@ -40,6 +40,7 @@
4040 authors?: unknown;
4141 autofix?: unknown;
4242 daily_limit?: unknown;
43+ tokens_per_review?: unknown;
4344 accept?: unknown;
4445 }
4546
@@ -80,6 +81,8 @@
8081 }
8182 const dailyLimit = input.daily_limit === undefined || input.daily_limit === "" ? REPO_DAILY_DEFAULT : Number(input.daily_limit);
8283 if (!Number.isInteger(dailyLimit) || dailyLimit < 1 || dailyLimit > REPO_DAILY_MAX) throw new HttpError(422, `\`daily_limit\` must be a whole number from 1 to ${REPO_DAILY_MAX}`);
84+ const tokensPerReview = input.tokens_per_review === undefined || input.tokens_per_review === "" ? REVIEW_TOKENS_DEFAULT : Number(input.tokens_per_review);
85+ if (!Number.isInteger(tokensPerReview) || tokensPerReview < REVIEW_TOKENS_MIN || tokensPerReview > REVIEW_TOKENS_MAX) throw new HttpError(422, `\`tokens_per_review\` must be a whole number from ${REVIEW_TOKENS_MIN} to ${REVIEW_TOKENS_MAX}`);
8386 if (input.accept !== true) throw new HttpError(422, "confirm that every agent review on this repository, including ones other people ask for, uses your model key");
8487 const at = now();
8588 const settings: ReviewSettings = {
@@ -93,6 +96,7 @@
9396 authors,
9497 autofix: input.autofix === true,
9598 dailyLimit,
99+ tokensPerReview,
96100 optedInAt: at,
97101 updatedAt: at,
98102 };
@@ -216,6 +220,7 @@
216220 authors: s.authors,
217221 autofix: s.autofix,
218222 daily_limit: s.dailyLimit ?? REPO_DAILY_DEFAULT,
223+ tokens_per_review: s.tokensPerReview ?? REVIEW_TOKENS_DEFAULT,
219224 opted_in_at: s.optedInAt,
220225 updated_at: s.updatedAt,
221226 }
modifiedapps/diffinery/src/agentic/tasks.ts+1−1View file
@@ -188,7 +188,7 @@
188188 html_url: `${base}/${repoFull}/tasks/${t.number}`,
189189 created_at: t.created_at,
190190 updated_at: t.updated_at,
191- ...("runs" in t ? { runs: t.runs.map((r) => ({ number: r.seq, kind: r.kind, message: r.message, state: r.step, exit_code: r.exit_code, error: r.error, started_at: r.started_at, ended_at: r.ended_at })) } : {}),
191+ ...("runs" in t ? { runs: t.runs.map((r) => ({ number: r.seq, kind: r.kind, message: r.message, state: r.step, exit_code: r.exit_code, error: r.error, model_requests: r.model_calls ?? 0, tokens: r.tokens ?? 0, started_at: r.started_at, ended_at: r.ended_at })) } : {}),
192192 };
193193 }
194194
addedapps/diffinery/src/agentic/usage.ts+185−0View file
@@ -1,0 +1,185 @@
1+/**
2+ * Token usage from model responses, read by the model proxy as they stream
3+ * through it, so a run's token budget is charged by Diffinery and not trusted
4+ * to the agent. Understands Anthropic Messages, OpenAI Responses and Chat
5+ * Completions, streamed (SSE) or not.
6+ */
7+
8+export interface Usage {
9+ /** Input tokens read without a cache. */
10+ input: number;
11+ output: number;
12+ cacheRead: number;
13+ cacheWrite: number;
14+}
15+
16+export const NO_USAGE: Usage = { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 };
17+
18+/**
19+ * Tokens as a budget counts them: cache reads cost about a tenth of other
20+ * input with every provider, so they count as a tenth.
21+ */
22+export const weighted = (u: Usage): number => u.input + u.cacheWrite + u.output + Math.ceil(u.cacheRead / 10);
23+
24+const n = (v: unknown): number => (typeof v === "number" && Number.isFinite(v) && v > 0 ? Math.floor(v) : 0);
25+
26+/** The usage one response object (or stream event) reports, if any. */
27+export function usageOf(o: unknown): Usage | null {
28+ if (!o || typeof o !== "object") return null;
29+ const x = o as Record<string, unknown>;
30+ const u = (x.usage ?? (x.message as Record<string, unknown> | undefined)?.usage ?? (x.response as Record<string, unknown> | undefined)?.usage) as Record<string, unknown> | undefined;
31+ if (!u || typeof u !== "object") return null;
32+ if ("prompt_tokens" in u || "completion_tokens" in u) {
33+ // Chat Completions: prompt_tokens includes cached ones.
34+ const cached = n((u.prompt_tokens_details as Record<string, unknown> | undefined)?.cached_tokens);
35+ return { input: Math.max(n(u.prompt_tokens) - cached, 0), output: n(u.completion_tokens), cacheRead: cached, cacheWrite: 0 };
36+ }
37+ const details = u.input_tokens_details as Record<string, unknown> | undefined;
38+ if (details && "cached_tokens" in details) {
39+ // OpenAI Responses: input_tokens includes cached ones.
40+ const cached = n(details.cached_tokens);
41+ return { input: Math.max(n(u.input_tokens) - cached, 0), output: n(u.output_tokens), cacheRead: cached, cacheWrite: 0 };
42+ }
43+ // Anthropic: input_tokens excludes cache reads and writes.
44+ return { input: n(u.input_tokens), output: n(u.output_tokens), cacheRead: n(u.cache_read_input_tokens), cacheWrite: n(u.cache_creation_input_tokens) };
45+}
46+
47+/** Events of one response report running totals (Anthropic's message_start, then message_delta): keep the largest of each. */
48+const fold = (a: Usage | null, b: Usage | null): Usage | null => (!a ? b : b ? { input: Math.max(a.input, b.input), output: Math.max(a.output, b.output), cacheRead: Math.max(a.cacheRead, b.cacheRead), cacheWrite: Math.max(a.cacheWrite, b.cacheWrite) } : a);
49+
50+const parse = (s: string): unknown => {
51+ try {
52+ return JSON.parse(s);
53+ } catch {
54+ return null;
55+ }
56+};
57+
58+/** The most of a non-streamed body that is read for its usage. */
59+const MAX_JSON = 16 * 1024 * 1024;
60+
61+/** An SSE line longer than this can't be a usage event: it is dropped rather than buffered. */
62+const MAX_SSE_LINE = 1024 * 1024;
63+
64+/** Read a response body (a tee of what goes back to the agent) for the usage it reports; null when it reports none. */
65+export async function readUsage(body: ReadableStream<Uint8Array>, contentType: string): Promise<Usage | null> {
66+ const sse = contentType.includes("text/event-stream");
67+ const reader = body.pipeThrough(new TextDecoderStream()).getReader();
68+ let usage: Usage | null = null;
69+ let buf = "";
70+ let size = 0;
71+ try {
72+ for (;;) {
73+ const { done, value } = await reader.read();
74+ if (done) break;
75+ if (!sse) {
76+ size += value.length;
77+ if (size <= MAX_JSON) buf += value;
78+ continue;
79+ }
80+ buf += value;
81+ let nl: number;
82+ while ((nl = buf.indexOf("\n")) >= 0) {
83+ const line = buf.slice(0, nl).trim();
84+ buf = buf.slice(nl + 1);
85+ if (line.startsWith("data:")) usage = fold(usage, usageOf(parse(line.slice(5).trim())));
86+ }
87+ if (buf.length > MAX_SSE_LINE) buf = "";
88+ }
89+ } finally {
90+ reader.releaseLock();
91+ }
92+ if (sse) return buf.trim().startsWith("data:") ? fold(usage, usageOf(parse(buf.trim().slice(5).trim()))) : usage;
93+ return size <= MAX_JSON ? fold(usage, usageOf(parse(buf))) : usage;
94+}
95+
96+export const formatTokens = (t: number): string => (t >= 1_000_000 ? `${+(t / 1_000_000).toFixed(1)}M` : t >= 1000 ? `${Math.round(t / 1000)}k` : String(t));
97+
98+/** Answers that generate tokens, and so are charged an estimate when they report no usage. */
99+const GENERATES = /^\/v1\/(messages|responses|responses\/compact|chat\/completions)$/;
100+/** The output an estimate assumes when a request doesn't cap it. */
101+export const DEFAULT_MAX_OUTPUT = 32_000;
102+
103+export interface PreparedRequest {
104+ /** The body to forward (changed only to ask for usage). */
105+ body: string;
106+ /** What the request may cost at most, booked before it goes upstream; null for requests that generate nothing. */
107+ estimate: Usage | null;
108+ /** Why a review may not send this request, if it may not. */
109+ reviewRefuse?: string;
110+ /** The body a review sends instead, when the request doesn't cap its output: capped, so the estimate holds. */
111+ reviewBody?: string;
112+}
113+
114+/** Where each generating endpoint takes its output cap. */
115+const CAP_FIELD: Record<string, string> = { "/v1/messages": "max_tokens", "/v1/responses": "max_output_tokens", "/v1/chat/completions": "max_completion_tokens" };
116+
117+/** Responses API fields that pull in context stored at the provider, which the request's size doesn't show. */
118+const STORED_CONTEXT = ["previous_response_id", "conversation", "prompt"] as const;
119+
120+/**
121+ * Look at a model request before it goes upstream, so that every answer can
122+ * be charged: its estimate (a token per 3 bytes of request, plus the output
123+ * it allows) is booked up front and settled to the usage the answer reports,
124+ * and streamed Chat Completions are asked to include usage. A review may not
125+ * start a background response, which it could leave unfetched and so
126+ * uncharged, pull in stored context its estimate can't see, or ask for more
127+ * than one answer; and a review's request that doesn't cap its output is
128+ * capped at the estimate's DEFAULT_MAX_OUTPUT.
129+ */
130+export function prepareRequest(method: string, path: string, body: string, size: number): PreparedRequest {
131+ if (method !== "POST" || !GENERATES.test(path)) return { body, estimate: null };
132+ const json = parse(body);
133+ const x = json && typeof json === "object" && !Array.isArray(json) ? (json as Record<string, unknown>) : null;
134+ const cap = x ? [x.max_tokens, x.max_output_tokens, x.max_completion_tokens].map(n).find((v) => v > 0) : undefined;
135+ const estimate: Usage = { input: Math.ceil(size / 3), output: cap ?? DEFAULT_MAX_OUTPUT, cacheRead: 0, cacheWrite: 0 };
136+ if (!x) return { body, estimate };
137+ const stored = path.startsWith("/v1/responses") ? STORED_CONTEXT.find((k) => x[k] != null) : undefined;
138+ const reviewRefuse =
139+ x.background === true
140+ ? "background responses aren't available to agent reviews"
141+ : stored
142+ ? `${stored} isn't available to agent reviews: send the whole input instead`
143+ : x.n != null && x.n !== 1
144+ ? "more than one answer per request (n) isn't available to agent reviews"
145+ : undefined;
146+ let fields = x;
147+ if (path === "/v1/chat/completions" && x.stream === true) {
148+ const so = x.stream_options && typeof x.stream_options === "object" ? (x.stream_options as Record<string, unknown>) : {};
149+ if (so.include_usage !== true) fields = { ...x, stream_options: { ...so, include_usage: true } };
150+ }
151+ const out: PreparedRequest = { body: fields === x ? body : JSON.stringify(fields), estimate };
152+ if (reviewRefuse) out.reviewRefuse = reviewRefuse;
153+ const field = CAP_FIELD[path];
154+ if (cap === undefined && field) out.reviewBody = JSON.stringify({ ...fields, [field]: DEFAULT_MAX_OUTPUT });
155+ return out;
156+}
157+
158+/** Read a request body up to `max` bytes, counting as it reads; null when it is larger. */
159+export async function readBody(body: ReadableStream<Uint8Array> | null, max: number): Promise<Uint8Array | null> {
160+ if (!body) return new Uint8Array();
161+ const chunks: Uint8Array[] = [];
162+ let size = 0;
163+ const reader = body.getReader();
164+ try {
165+ for (;;) {
166+ const { done, value } = await reader.read();
167+ if (done) break;
168+ size += value.byteLength;
169+ if (size > max) {
170+ await reader.cancel().catch(() => undefined);
171+ return null;
172+ }
173+ chunks.push(value);
174+ }
175+ } finally {
176+ reader.releaseLock();
177+ }
178+ const out = new Uint8Array(size);
179+ let at = 0;
180+ for (const c of chunks) {
181+ out.set(c, at);
182+ at += c.byteLength;
183+ }
184+ return out;
185+}
modifiedapps/diffinery/src/docs/api-meta.ts+3−2View file
@@ -1405,7 +1405,7 @@
14051405 },
14061406 "GET /repos/{owner}/{repo}/diffinery/agent-review": {
14071407 summary: "Get agent review settings",
1408- description: "Whether agent review is on, which agent reviews with which runtime, whose model key every review uses (`uses_key_of`), when it reviews (`trigger`: `push`, `open` or `request`), which pull requests (`authors`: `all`, `people` or `agents`), `drafts`, `autofix` and `daily_limit`. `{\"enabled\": false}` when it was never turned on. Needs write access. See /docs/agents/review.",
1408+ description: "Whether agent review is on, which agent reviews with which runtime, whose model key every review uses (`uses_key_of`), when it reviews (`trigger`: `push`, `open` or `request`), which pull requests (`authors`: `all`, `people` or `agents`), `drafts`, `autofix`, `daily_limit` and `tokens_per_review`. `{\"enabled\": false}` when it was never turned on. Needs write access. See /docs/agents/review.",
14091409 access: "write",
14101410 },
14111411 "PUT /repos/{owner}/{repo}/diffinery/agent-review": {
@@ -1420,7 +1420,8 @@
14201420 authors: ["string", "`all` (default), `people` or `agents`."],
14211421 drafts: ["boolean", "Review drafts too (default false)."],
14221422 autofix: ["boolean", "Start a fix task on the pull request's branch for blocking findings (default false; at most 3 times per pull request)."],
1423- daily_limit: ["integer", "Most reviews the repository runs in 24 hours, 1 to 1000 (default 50). Past it, pull requests get an action-required check and you get an email."],
1423+ daily_limit: ["integer", "Most reviews and autofix runs the repository starts in 24 hours, 1 to 1000 (default 50). Past it, pull requests get an action-required check, autofix waits, and you get an email."],
1424+ tokens_per_review: ["integer", "Most tokens one review may use, counted from the provider's responses (cache reads count as a tenth), 100000 to 50000000 (default 2000000)."],
14241425 accept_terms: ["boolean", "Must be true: reviews use your model key."],
14251426 enabled: ["boolean", "`false` turns agent review off."],
14261427 },
modifiedapps/diffinery/test/e2e/review.test.mjs+115−3View file
@@ -111,8 +111,33 @@
111111 req.on("data", (d) => (body += d));
112112 req.on("end", () => {
113113 modelRequests.push({ method: req.method, url: req.url, headers: req.headers, body });
114+ if (body.includes('"abort":true')) {
115+ // A streamed answer cut off before it reports usage.
116+ res.writeHead(200, { "content-type": "text/event-stream" });
117+ res.write(`event: content_block_delta\ndata: ${JSON.stringify({ type: "content_block_delta", delta: { type: "text_delta", text: "cut" } })}\n\n`);
118+ setTimeout(() => res.destroy(), 100);
119+ return;
120+ }
121+ if (body.includes('"max_tokens":7')) {
122+ // An answer that reports no usage.
123+ res.writeHead(200, { "content-type": "application/json" });
124+ res.end(JSON.stringify({ type: "message", content: [{ type: "text", text: "no usage here" }] }));
125+ return;
126+ }
127+ if (body.includes('"stream":true')) {
128+ // A streamed answer whose usage adds up to 40,000 tokens: 1,000 in, 39,000 out ("slow" ones take a moment).
129+ const send = () => {
130+ res.writeHead(200, { "content-type": "text/event-stream" });
131+ res.write(`event: message_start\ndata: ${JSON.stringify({ type: "message_start", message: { usage: { input_tokens: 1000, output_tokens: 1 } } })}\n\n`);
132+ res.write(`event: content_block_delta\ndata: ${JSON.stringify({ type: "content_block_delta", delta: { type: "text_delta", text: "fake model says hi" } })}\n\n`);
133+ res.end(`event: message_delta\ndata: ${JSON.stringify({ type: "message_delta", usage: { output_tokens: 39000 } })}\n\nevent: message_stop\ndata: {"type":"message_stop"}\n\n`);
134+ };
135+ if (body.includes('"slow":true')) setTimeout(send, 1500);
136+ else send();
137+ return;
138+ }
114139 res.writeHead(200, { "content-type": "application/json", "set-cookie": "upstream=1" });
115- res.end(JSON.stringify({ type: "message", content: [{ type: "text", text: "fake model says hi" }] }));
140+ res.end(JSON.stringify({ type: "message", content: [{ type: "text", text: "fake model says hi" }], usage: { input_tokens: 10, output_tokens: 5 } }));
116141 });
117142 });
118143 await new Promise((r) => upstream.listen(0, "127.0.0.1", r));
@@ -385,7 +410,7 @@
385410 assert.match(off.json.message, /isn't turned on/);
386411 });
387412
388-test("agent review: spend limits per review and per repository", async () => {
413+test("agent review: spend limits per review (requests, tokens) and per repository", async () => {
389414 const owner = (await ok("GET", "/user")).login;
390415 const S = `/repos/${owner}/review-demo/diffinery/agent-review`;
391416 const P = `/repos/${owner}/review-demo/pulls/1/diffinery/agent-review`;
@@ -418,7 +443,94 @@
418443 const head = (await ok("GET", `/repos/${owner}/review-demo/pulls/1`)).head.sha;
419444 const check = () => ok("GET", `/repos/${owner}/review-demo/commits/${head}/check-runs`).then((r) => r.check_runs.find((c) => c.name === "Agent review"));
420445 assert.equal((await check()).conclusion, "failure");
446+ assert.equal(run.model_requests, 201);
447+ const runTokens = async (n) => (await ok("GET", `/repos/${owner}/review-demo/diffinery/tasks/${task}`)).runs.find((r) => r.number === n).tokens;
448+ // Each answer is charged once it has streamed through, so the last one may land a moment later.
449+ await until("every answer's usage to be counted", async () => (await runTokens(run.number)) === 200 * 15, 15_000);
421450
451+ // Per review: the token budget, counted from the usage streamed answers report.
452+ const streamy = join(work, "streamy-agent.sh");
453+ writeFileSync(streamy, `#!/bin/bash\nfor i in $(seq 1 20); do\n curl -s -o /dev/null -X POST "$ANTHROPIC_BASE_URL/v1/messages" -H "x-api-key: $ANTHROPIC_API_KEY" -H "content-type: application/json" -d '{"model":"m","stream":true,"messages":[]}'\n sleep 0.3\ndone\n`);
454+ for (const tokens_per_review of [99_999, 50_000_001]) assert.equal((await asPerson("PUT", S, { ...settings, daily_limit: 1000, tokens_per_review })).status, 422);
455+ const budget = await asPerson("PUT", S, { ...settings, command: `bash ${streamy}`, daily_limit: 1000, tokens_per_review: 100_000 });
456+ assert.equal(budget.json.tokens_per_review, 100_000);
457+ const before2 = modelRequests.length;
458+ assert.equal((await api("POST", P, {})).status, 202);
459+ const t2 = await until("the review to reach its token budget", async () => {
460+ const t = await ok("GET", `/repos/${owner}/review-demo/diffinery/tasks/${task}`);
461+ return t.runs.length > runs0 + 1 && t.state !== "running" ? t : null;
462+ }, 180_000);
463+ const run2 = t2.runs.at(-1);
464+ assert.equal(run2.state, "failed");
465+ // Each request books the most it may cost (32,000 output when it doesn't cap it) before it is sent, so the third is refused.
466+ assert.match(run2.error, /next model request could take it past its budget of 100k tokens \(80k used\), so it was stopped/);
467+ assert.equal(modelRequests.length - before2, 2);
468+ await until("the streamed answers' usage to be counted", async () => (await runTokens(run2.number)) === 2 * 40_000, 15_000);
469+
470+ // Requests sent at once can't overrun the budget together: each is booked before it goes upstream.
471+ const burst = join(work, "burst-agent.sh");
472+ writeFileSync(
473+ burst,
474+ `#!/bin/bash\nfor i in $(seq 1 20); do\n curl -s -o /dev/null -w "%{http_code}\\n" -X POST "$ANTHROPIC_BASE_URL/v1/messages" -H "x-api-key: $ANTHROPIC_API_KEY" -H "content-type: application/json" -d '{"model":"m","max_tokens":39000,"slow":true,"stream":true,"messages":[]}' &\ndone\nwait\n`,
475+ );
476+ assert.equal((await asPerson("PUT", S, { ...settings, command: `bash ${burst}`, daily_limit: 1000, tokens_per_review: 100_000 })).status, 200);
477+ const before4 = modelRequests.length;
478+ assert.equal((await api("POST", P, {})).status, 202);
479+ const t4 = await until("the burst review to stop", async () => {
480+ const t = await ok("GET", `/repos/${owner}/review-demo/diffinery/tasks/${task}`);
481+ return t.runs.length > runs0 + 2 && t.state !== "running" ? t : null;
482+ }, 180_000);
483+ const run4 = t4.runs.at(-1);
484+ assert.equal(run4.state, "failed");
485+ assert.match(run4.error, /could take it past its budget of 100k tokens/);
486+ // Two bookings of 39,024 fit in 100k, a third doesn't.
487+ await until("the burst's answers to finish", async () => modelRequests.length - before4 === 2, 15_000);
488+ await new Promise((r) => setTimeout(r, 2000));
489+ assert.equal(modelRequests.length - before4, 2, "only two of the twenty requests reached the provider");
490+ await until("the burst's usage to be settled", async () => (await runTokens(run4.number)) === 2 * 40_000, 15_000);
491+
492+ // Every answer is charged: one without usage costs an estimate, streamed Chat Completions are asked for usage,
493+ // and a review can't start a background response it would never fetch.
494+ const OPENAI_KEY = `sk-proj-${randomBytes(30).toString("hex")}`;
495+ assert.equal((await asPerson("PUT", "/diffinery/model-keys/openai", { key: OPENAI_KEY, accept_terms: true })).status, 200);
496+ const probe = join(work, "usage-agent.sh");
497+ const CUT = '{"model":"m","max_tokens":500,"abort":true,"messages":[]}';
498+ writeFileSync(
499+ probe,
500+ [
501+ "#!/bin/bash",
502+ `curl -s -o /dev/null -X POST "$ANTHROPIC_BASE_URL/v1/messages" -H "x-api-key: $ANTHROPIC_API_KEY" -H "content-type: application/json" -d '{"model":"m","max_tokens":7,"messages":[]}'`,
503+ `echo "background $(curl -s -o /dev/null -w '%{http_code}' -X POST "$OPENAI_BASE_URL/responses" -H "authorization: Bearer $OPENAI_API_KEY" -H "content-type: application/json" -d '{"model":"m","background":true,"input":"x"}')"`,
504+ `curl -s -o /dev/null -X POST "$OPENAI_BASE_URL/chat/completions" -H "authorization: Bearer $OPENAI_API_KEY" -H "content-type: application/json" -d '{"model":"m","stream":true,"messages":[]}'`,
505+ ...["previous_response_id", "conversation", "prompt"].map(
506+ (k) => `echo "${k} $(curl -s -o /dev/null -w '%{http_code}' -X POST "$OPENAI_BASE_URL/responses" -H "authorization: Bearer $OPENAI_API_KEY" -H "content-type: application/json" -d '{"model":"m","${k}":"x","input":"x"}')"`,
507+ ),
508+ `echo "n $(curl -s -o /dev/null -w '%{http_code}' -X POST "$OPENAI_BASE_URL/chat/completions" -H "authorization: Bearer $OPENAI_API_KEY" -H "content-type: application/json" -d '{"model":"m","n":128,"max_completion_tokens":16000,"messages":[]}')"`,
509+ `curl -s -o /dev/null -X POST "$ANTHROPIC_BASE_URL/v1/messages" -H "x-api-key: $ANTHROPIC_API_KEY" -H "content-type: application/json" -d '${CUT}'`,
510+ "sleep 1",
511+ "",
512+ ].join("\n"),
513+ );
514+ assert.equal((await asPerson("PUT", S, { ...settings, command: `bash ${probe}`, daily_limit: 1000 })).status, 200);
515+ const before3 = modelRequests.length;
516+ assert.equal((await api("POST", P, {})).status, 202);
517+ const t3 = await until("the usage probe review", async () => {
518+ const t = await ok("GET", `/repos/${owner}/review-demo/diffinery/tasks/${task}`);
519+ return t.runs.length > runs0 + 3 && t.state !== "running" ? t : null;
520+ }, 180_000);
521+ const run3 = t3.runs.at(-1);
522+ const text3 = (await ok("GET", `/repos/${owner}/review-demo/diffinery/tasks/${task}/transcript`)).filter((l) => l.run === run3.number).map((l) => l.text).join("\n");
523+ for (const k of ["background", "previous_response_id", "conversation", "prompt", "n"]) assert.match(text3, new RegExp(`^${k} 403$`, "m"), text3);
524+ const served3 = modelRequests.slice(before3);
525+ assert.equal(served3.length, 3, "the refused requests never reached the provider");
526+ // A streamed Chat Completion is asked for usage, and one with no output cap is capped at what its estimate assumed.
527+ assert.equal(JSON.parse(served3[1].body).stream_options.include_usage, true);
528+ assert.equal(JSON.parse(served3[1].body).max_completion_tokens, 32_000);
529+ // 42 bytes / 3 + max_tokens 7 for the answer without usage, the streamed answer's own 40,000,
530+ // and the estimate the cut-off answer was booked, which stays charged since its usage was never read.
531+ const cut = Math.ceil(Buffer.byteLength(CUT) / 3) + 500;
532+ await until("the estimates and the streamed usage to be counted", async () => (await runTokens(run3.number)) === 14 + 7 + 40_000 + cut, 15_000);
533+
422534 // Per repository: past the payer's daily limit, a request is refused, the check asks for action, and the payer gets one email.
423535 assert.equal((await asPerson("PUT", S, { ...settings, daily_limit: 1 })).status, 200);
424536 const limited = await api("POST", P, {});
@@ -433,5 +545,5 @@
433545 assert.equal((await api("POST", P, {})).status, 429);
434546 await sleep(3000);
435547 assert.equal((await mails()).length, 1, "one email a day");
436- assert.equal((await ok("GET", `/repos/${owner}/review-demo/diffinery/tasks/${task}`)).runs.length, runs0 + 1, "nothing ran past the limit");
548+ assert.equal((await ok("GET", `/repos/${owner}/review-demo/diffinery/tasks/${task}`)).runs.length, runs0 + 4, "nothing ran past the limit");
437549 });
addedapps/diffinery/test/unit/usage.test.ts+102−0View file
@@ -1,0 +1,102 @@
1+import { describe, expect, it } from "vitest";
2+import { DEFAULT_MAX_OUTPUT, formatTokens, prepareRequest, readBody, readUsage, usageOf, weighted } from "../../src/agentic/usage.js";
3+
4+const stream = (chunks: string[]) =>
5+ new ReadableStream<Uint8Array>({
6+ start(c) {
7+ for (const s of chunks) c.enqueue(new TextEncoder().encode(s));
8+ c.close();
9+ },
10+ });
11+
12+describe("usageOf", () => {
13+ it("reads Anthropic, OpenAI Responses and Chat Completions usage", () => {
14+ expect(usageOf({ usage: { input_tokens: 10, output_tokens: 5, cache_read_input_tokens: 100, cache_creation_input_tokens: 20 } })).toEqual({ input: 10, output: 5, cacheRead: 100, cacheWrite: 20 });
15+ expect(usageOf({ type: "message_start", message: { usage: { input_tokens: 7, output_tokens: 1 } } })).toEqual({ input: 7, output: 1, cacheRead: 0, cacheWrite: 0 });
16+ expect(usageOf({ type: "response.completed", response: { usage: { input_tokens: 120, input_tokens_details: { cached_tokens: 100 }, output_tokens: 9 } } })).toEqual({ input: 20, output: 9, cacheRead: 100, cacheWrite: 0 });
17+ expect(usageOf({ usage: { prompt_tokens: 50, completion_tokens: 4, prompt_tokens_details: { cached_tokens: 40 } } })).toEqual({ input: 10, output: 4, cacheRead: 40, cacheWrite: 0 });
18+ expect(usageOf({ type: "ping" })).toBeNull();
19+ expect(usageOf({ usage: { input_tokens: -3, output_tokens: "9" } })).toEqual({ input: 0, output: 0, cacheRead: 0, cacheWrite: 0 });
20+ });
21+ it("weighs cache reads as a tenth", () => {
22+ expect(weighted({ input: 10, output: 5, cacheRead: 101, cacheWrite: 20 })).toBe(10 + 5 + 20 + 11);
23+ expect(formatTokens(2_000_000)).toBe("2M");
24+ expect(formatTokens(120_000)).toBe("120k");
25+ expect(formatTokens(950)).toBe("950");
26+ });
27+});
28+
29+describe("readUsage", () => {
30+ it("adds up an Anthropic stream's running totals, split across chunks", async () => {
31+ const sse = [
32+ 'event: message_start\ndata: {"type":"message_start","message":{"usage":{"input_tokens":1000,"cache_read_input_tokens":5000,"output_tokens":1}}}\n\n',
33+ 'event: content_block_delta\ndata: {"type":"content_block_delta","delta":{"text":"hi"}}\n\nevent: message_delta\nda',
34+ 'ta: {"type":"message_delta","usage":{"output_tokens":300}}\n\nevent: message_stop\ndata: {"type":"message_stop"}',
35+ ];
36+ expect(await readUsage(stream(sse), "text/event-stream; charset=utf-8")).toEqual({ input: 1000, output: 300, cacheRead: 5000, cacheWrite: 0 });
37+ });
38+ it("reads an OpenAI Responses stream and a plain JSON answer", async () => {
39+ const sse = ['data: {"type":"response.created","response":{}}\n\n', 'data: {"type":"response.completed","response":{"usage":{"input_tokens":80,"input_tokens_details":{"cached_tokens":0},"output_tokens":12}}}\n\n'];
40+ expect(await readUsage(stream(sse), "text/event-stream")).toEqual({ input: 80, output: 12, cacheRead: 0, cacheWrite: 0 });
41+ expect(await readUsage(stream(['{"id":"x","usage":{"prompt_tokens":3,', '"completion_tokens":2}}']), "application/json")).toEqual({ input: 3, output: 2, cacheRead: 0, cacheWrite: 0 });
42+ expect(await readUsage(stream(["not json"]), "application/json")).toBeNull();
43+ expect(await readUsage(stream(['data: {"choices":[{"delta":{"content":"hi"}}]}\n\n', "data: [DONE]\n\n"]), "text/event-stream")).toBeNull();
44+ });
45+ it("drops an SSE line too long to be a usage event, and still reads the rest", async () => {
46+ const huge = "data: " + "x".repeat(1100 * 1024);
47+ expect(await readUsage(stream([huge.slice(0, 600_000), huge.slice(600_000), '\ndata: {"usage":{"input_tokens":4,"output_tokens":2}}\n\n']), "text/event-stream")).toEqual({ input: 4, output: 2, cacheRead: 0, cacheWrite: 0 });
48+ });
49+});
50+
51+describe("prepareRequest", () => {
52+ const prep = (method: string, path: string, body: string) => prepareRequest(method, path, body, new TextEncoder().encode(body).length);
53+ it("estimates the most an answer may cost: a token per 3 bytes plus the output it allows", () => {
54+ const body = JSON.stringify({ model: "m", max_tokens: 1000, messages: [{ role: "user", content: "é".repeat(300) }] });
55+ expect(prep("POST", "/v1/messages", body)).toEqual({ body, estimate: { input: Math.ceil((body.length + 300) / 3), output: 1000, cacheRead: 0, cacheWrite: 0 } });
56+ expect(prep("POST", "/v1/responses", '{"max_output_tokens":50}').estimate?.output).toBe(50);
57+ expect(prep("POST", "/v1/chat/completions", '{"max_completion_tokens":70}').estimate?.output).toBe(70);
58+ expect(prep("POST", "/v1/messages", '{"model":"m"}').estimate?.output).toBe(DEFAULT_MAX_OUTPUT);
59+ expect(prep("POST", "/v1/messages", "not json").estimate).toEqual({ input: 3, output: DEFAULT_MAX_OUTPUT, cacheRead: 0, cacheWrite: 0 });
60+ // Requests that generate nothing are never estimated.
61+ expect(prep("POST", "/v1/messages/count_tokens", "{}").estimate).toBeNull();
62+ expect(prep("GET", "/v1/models", "").estimate).toBeNull();
63+ });
64+ it("asks streamed Chat Completions for usage", () => {
65+ const p = prep("POST", "/v1/chat/completions", '{"model":"m","stream":true,"stream_options":{"include_usage":false,"x":1}}');
66+ expect(JSON.parse(p.body).stream_options).toEqual({ include_usage: true, x: 1 });
67+ expect(JSON.parse(prep("POST", "/v1/chat/completions", '{"stream":true}').body).stream_options).toEqual({ include_usage: true });
68+ const unchanged = '{"stream":false}';
69+ expect(prep("POST", "/v1/chat/completions", unchanged).body).toBe(unchanged);
70+ });
71+ it("names what a review may not send: background responses and context stored at the provider", () => {
72+ expect(prep("POST", "/v1/responses", '{"background":true}').reviewRefuse).toMatch(/background/);
73+ expect(prep("POST", "/v1/responses", '{"previous_response_id":"resp_1","input":"x"}').reviewRefuse).toMatch(/^previous_response_id/);
74+ expect(prep("POST", "/v1/responses/compact", '{"conversation":"conv_1"}').reviewRefuse).toMatch(/^conversation/);
75+ expect(prep("POST", "/v1/responses", '{"prompt":{"id":"pmpt_1"}}').reviewRefuse).toMatch(/^prompt/);
76+ expect(prep("POST", "/v1/responses", '{"previous_response_id":null,"input":"x"}').reviewRefuse).toBeUndefined();
77+ expect(prep("POST", "/v1/messages", '{"prompt":"x","max_tokens":5}').reviewRefuse).toBeUndefined();
78+ expect(prep("POST", "/v1/chat/completions", '{"n":128,"max_completion_tokens":16000}').reviewRefuse).toMatch(/\(n\)/);
79+ expect(prep("POST", "/v1/chat/completions", '{"n":"2"}').reviewRefuse).toMatch(/\(n\)/);
80+ expect(prep("POST", "/v1/chat/completions", '{"n":1,"max_completion_tokens":5}').reviewRefuse).toBeUndefined();
81+ });
82+ it("gives reviews a body capped at the estimate's output when the request sets no cap", () => {
83+ expect(JSON.parse(prep("POST", "/v1/responses", '{"input":"x"}').reviewBody!)).toEqual({ input: "x", max_output_tokens: DEFAULT_MAX_OUTPUT });
84+ expect(JSON.parse(prep("POST", "/v1/messages", '{"messages":[]}').reviewBody!)).toEqual({ messages: [], max_tokens: DEFAULT_MAX_OUTPUT });
85+ const streamed = JSON.parse(prep("POST", "/v1/chat/completions", '{"stream":true}').reviewBody!);
86+ expect(streamed).toEqual({ stream: true, stream_options: { include_usage: true }, max_completion_tokens: DEFAULT_MAX_OUTPUT });
87+ // A request that caps its output, any way the endpoint takes it, is sent as it is.
88+ expect(prep("POST", "/v1/chat/completions", '{"max_tokens":9}').reviewBody).toBeUndefined();
89+ expect(prep("POST", "/v1/responses", '{"max_output_tokens":9}').reviewBody).toBeUndefined();
90+ expect(prep("POST", "/v1/messages/count_tokens", "{}").reviewBody).toBeUndefined();
91+ });
92+});
93+
94+describe("readBody", () => {
95+ const chunks = (parts: Uint8Array[]) => new ReadableStream<Uint8Array>({ start(c) { for (const p of parts) c.enqueue(p); c.close(); } });
96+ it("reads a body whole, and refuses one past the limit by its bytes", async () => {
97+ const e = new TextEncoder();
98+ expect(new TextDecoder().decode((await readBody(chunks([e.encode("ab"), e.encode("çd")]), 5))!)).toBe("abçd");
99+ expect(await readBody(chunks([e.encode("ab"), e.encode("çde")]), 5)).toBeNull();
100+ expect(await readBody(null, 5)).toEqual(new Uint8Array());
101+ });
102+});