Diffinery

codician/diffinerypublic

Open-source, agent-first GitHub and GitHub Actions alternative that runs on your own Cloudflare account: git hosting, pull requests, code review and drop-in GitHub Actions YAML on Workers, Durable Objects, R2 and Containers.

Organizations' agent policy and model keys: whose key pays, what agents may use, and a monthly allowance (phase 5b) (#160)

* WIP: organizations' agent policies and model keys (phase 5b)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014VDQqT8oL18uLJ9fpsNxW2

* Organizations' agent policies and model keys (phase 5b)

An organization's owners decide whose key pays for agent runs in its
repositories (each person's own, the organization's when it has them, or
only the organization's), which runtimes and providers agents may use, the
most tasks at once per repository, terminals, agent review, and a monthly
token allowance per person on the organization's keys. The model proxy
re-checks the payer on every request before it decrypts the key.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014VDQqT8oL18uLJ9fpsNxW2

* Hold runs on an organization's key to their bookings; terminal tasks follow the keys and provider policy; bounded estimates; Security's lows

- M1: runs paid by an organization's key may not use background responses,
  stored context, several answers or routing fields, and send their output
  capped as booked, so the monthly allowance holds.
- S1: a terminal's task is held to the allowed providers and, under
  'only the organization's keys', to members.
- S2: a request books at most 1,000,000 output tokens.
- L1: deleting an account keeps the runs an organization's key paid for.
- L2: a transfer sets the repository's limit on tasks at once; saving the
  policy reaches every repository.
- L3: a site administrator's policy change is audited as such.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014VDQqT8oL18uLJ9fpsNxW2

* Org-key and review runs: refuse input the request doesn't carry, strip paid tools, charge unreadable usage at most

Security's re-check of #160 (M1, input side):
- refuse item_reference inputs, an Anthropic container, and images, documents
  and files given by URL or file id on runs held to their bookings
- runs on an organization's key send the review body: output capped as booked
  and provider-side tools stripped
- an answer whose usage can't be read is charged a token per byte sent plus
  its output cap on those runs
- refuse any background but false, bodies that aren't JSON objects, and
  OpenRouter :online models
- docs say what the overshoot really is

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014VDQqT8oL18uLJ9fpsNxW2

* Limited runs: at most 4 model requests at once, inline media and nested calls, hosted shells, presets

Security's third look at #160:
- a review or budgeted run, and a person on an organization's keys under an
  allowance, has at most 4 generating requests on their way at once (D1 leases,
  model_in_flight in 0031), so images and PDFs, which bill by pixels and pages,
  can take them past a budget by at most that many requests; docs say so
- the input walk skips only a tool call's input or arguments, not the whole object
- a tool with a non-local environment (OpenAI's hosted shell) is stripped
- no usage at all is charged like unreadable usage on limited runs
- limited runs always send the body written out again (duplicate keys)
- OpenRouter presets and :online with further suffixes are refused
- reading stored responses is refused, and OpenAI organization and project
  headers are dropped on an organization's key

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014VDQqT8oL18uLJ9fpsNxW2

* Review e2e: an answer without usage is charged a token per byte sent

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014VDQqT8oL18uLJ9fpsNxW2

* Model proxy: forward and settle under one waitUntil registered before the fetch; presets in any form

Security's fourth look at #160:
- everything after the request is read (access checks, slots, the provider's
  answer and its settlement) runs in one promise registered with waitUntil
  first, so a sandbox that hangs up mid-request still has it settled and its
  slots freed, and a throw after a slot was taken frees it
- @preset/ anywhere in the model, in any case, is refused

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014VDQqT8oL18uLJ9fpsNxW2

---------

Co-authored-by: Claude <noreply@anthropic.com>
Canberk Demirci authored · GitHub committed signed1 parent 3cece21commit ffd58b3c3299675bae462f9b9d5c5212c998759cBrowse files
40 files changed+2037−185
Files (40)
modifiedapps/diffinery/docs/accounts/organizations.md+2−1View file
@@ -24,6 +24,7 @@
2424 | Manage organization Actions secrets | No | Yes | Yes |
2525 | Read and manage organization Actions variables | No | Yes | Yes |
2626 | Open the **Settings** tab: rename, display name, base permission, picture | No | No | Yes |
27+| Set the [agent policy](/docs/agents/organizations), and see or change the organization's model keys | No | No | Yes |
2728 | Transfer repositories into the organization | No | Yes | Yes |
2829 | Delete the organization (once it owns no repositories or images) | No | No | Yes |
2930 | Change the display name, base permission and picture through the API | No | No | Yes |
@@ -173,7 +174,7 @@
173174 | **Repositories** | Everyone signed in | Repositories you can see. Members get a **New repository** button. |
174175 | **People** | Everyone signed in | Members and their roles. Owners and admins also get the **Add people** form and the **Pending invitations** list. |
175176 | **Agents** | Everyone signed in | The organization's agents, their maximum role and status. Owners and admins get a **Manage agents** button. |
176-| **Settings** | Owners | The organization's picture, name, display name and [base permission](#base-permission). Others get `403 organization owners only`. |
177+| **Settings** | Owners | The organization's picture, name, display name and [base permission](#base-permission), and its [agent policy and model keys](/docs/agents/organizations). Others get `403 organization owners only`. |
177178 | **Packages** | Everyone signed in | Container images the organization owns (`/acme/-/packages`). See [Container registry](/docs/packages/container-registry). |
178179
179180 Members see every repository of the organization, unless the base permission is `none`: then they see public repositories and the ones they were given access to or created. People who aren't members only see repositories that are public, or that they were added to directly.
addedapps/diffinery/docs/agents/organizations.md+143−0View file
@@ -1,0 +1,143 @@
1+---
2+title: Agents in organizations
3+description: An organization's agent policy and model keys. Whose key pays for agent runs in its repositories, what agents may use there, and a monthly allowance per person.
4+---
5+
6+An organization's owners decide how agents work in its repositories, on the organization's **Settings** tab under **Agents**:
7+
8+- whose model key pays: each person's own, or the organization's;
9+- which runtimes and providers agents may use;
10+- how many tasks run at once per repository;
11+- whether web terminals and agent review are allowed;
12+- how many tokens each person may spend on the organization's keys in a month.
13+
14+Nothing changes until an owner changes something. By default people use their own keys, as in [Agent tasks](/docs/agents/tasks#model-access), and everything is allowed. A repository owned by a person has no policy.
15+
16+## The organization's model keys
17+
18+An owner can save an Anthropic, OpenAI or OpenRouter API key for the organization, under **Settings → Agents → Model keys**.
19+
20+- Saving a key asks the owner to accept the provider's terms for the organization. The provider bills the organization's account. The acceptance is recorded with its date and the owner who accepted.
21+- **Only owners** can see or change the organization's keys. Org admins can't, and neither can members.
22+- Saving or deleting a key needs a browser session with a sign-in from the last 2 hours. Diffinery asks you to sign in again if yours is older. No token can do it.
23+- Keys are **write-only**, like personal keys: pages and the API show only the provider, the last four characters and who accepted the terms when.
24+- Every owner gets a security email when a key is added ("An Anthropic API key was added to acme").
25+- Saves and deletions are in the audit log as `org.model_key.set` and `org.model_key.delete`. A [site administrator](/docs/admin/site-administration) can act as an owner, and is recorded as themselves.
26+
27+The organization's key **never enters a sandbox, a container or a terminal**. As with personal keys, a run gets only its own model token. Diffinery's model proxy decrypts the key inside each request it forwards, after checking that the request may use it (see [How your key is protected](/docs/agents/tasks#how-your-key-is-protected)).
28+
29+## Whose key pays
30+
31+| Setting | Who pays |
32+| --- | --- |
33+| **Each person's own keys** (default) | The person who starts the work, with their own key, as anywhere else. |
34+| **The organization's keys, when it has them** | A member's run uses the organization's key when it has every provider the run needs, otherwise the member's own. An outside collaborator always uses their own key. |
35+| **Only the organization's keys** | Personal keys are never used in the organization's repositories. Only members can start tasks, agent reviews and coordinators there. An outside collaborator gets `403 acme pays for agents in its repositories with its own keys, which only its members can use`. |
36+
37+**Only members spend the organization's key.** A member here is a person with the member, admin or owner role. An agent account never counts, even an agent the organization owns: what counts is the person behind the run.
38+
39+- A task's payer is the person who started it, never its agent. A task a coordinator starts is paid like the coordinator's own run.
40+- An agent review's payer is the admin who turned reviews on (see [Agent review](/docs/agents/review)).
41+- A coordinator's payer is the person whose coordinator it is (see [Projects](/docs/agents/projects)).
42+
43+**A task keeps whose key pays from when it starts.** The REST task shows it as `"paid_by": "organization"` or `"person"`.
44+
45+- Messages, retries and plan approval keep it. Only the person who started the task can do those.
46+- An agent review decides when each review starts, and a coordinator when each run starts.
47+- A task the organization paid for can't be retried by someone who is no longer a member: `only members of acme can run agents on its keys`.
48+
49+**Every model request checks again**, before the key is decrypted. Nothing is cached, so a change applies at the next request. The proxy checks that:
50+
51+1. the repository still belongs to the organization, so a repository transferred out of it mid-task stops spending its key;
52+2. the policy still lets the organization's keys pay;
53+3. the person is still an active member;
54+4. the provider is still allowed;
55+5. the organization still has the key;
56+6. the person is under their monthly allowance.
57+
58+For a personal key in an organization's repository, it checks that the policy doesn't now require the organization's keys, and that the provider is still allowed.
59+
60+When a check fails, the proxy answers `403` with the reason, for example `mia is no longer a member of acme, whose key this task uses`. The task stops with a note saying why, and anything booked for that request is refunded.
61+
62+## Monthly allowance per person
63+
64+**Monthly tokens per person on the organization's keys** caps how much one person's runs can spend of the organization's keys in a calendar month (UTC). It goes from 10,000 to 10 billion, and is off by default.
65+
66+- Tokens are counted as [run budgets](/docs/agents/projects#limits) count them: input, output and cache writes in full, and cache reads as a tenth.
67+- Before a request goes to the provider, the proxy books an estimate against the person's allowance, in one step: a token per 3 bytes of the request, plus the output it allows. The answer then settles the booking to what the provider reported.
68+- A request that would take the person over the allowance is refused with `403` and never reaches the provider: `this month's allowance of 2,000,000 tokens per person on acme's keys is used up`. The task stops.
69+- While an allowance is set, a person has at most 4 model requests on the organization's keys on their way at once, across all their runs. A fifth gets `429` with `retry-after`, and the agent's CLI tries again.
70+- So requests already on their way can take a person past the allowance by at most 4 requests. Each one's output is capped at what it booked; its input can cost more than its estimate when it tokenizes densely or carries images or PDFs, which bill by pixels and pages.
71+- A request still streaming after 15 minutes stops counting toward the 4.
72+- An answer that reports no usage, or whose usage can't be read (a streamed line too long to keep, for example), is charged a token per byte sent, plus its output cap.
73+- A request books at most 1,000,000 output tokens, whatever cap it asks for, and is sent with that cap.
74+- Runs on a person's own key don't count.
75+
76+So that a booking holds what a request can use, runs on the organization's key send only what their booking bounds, as [agent reviews](/docs/agents/review) do:
77+
78+- A request that sets no output cap is sent with the one it was booked for, 32,000 tokens.
79+- Requests are refused with `403` when they pull in input the request doesn't carry:
80+ - context stored at the provider: `previous_response_id`, `conversation`, `prompt` or `item_reference` inputs in the Responses API, and an Anthropic `container`;
81+ - images, documents and files given by URL or file id rather than as their content.
82+- So are requests that start a background response (any `background` but `false`), ask for more than one answer (`n`), use an OpenRouter web search model (`:online`) or preset (`@preset/...`), or whose body isn't a JSON object, and routing and paid extras (`models`, `route`, `provider`, `plugins`, `web_search_options`, `speed`, `best_of`, `preset`). Reading back a stored response (`GET /v1/responses/...`) is refused too. Claude Code and Codex don't send any of them in normal use.
83+- Tools that run on the provider (web search, web fetch, code execution, file search, MCP servers, a shell in a hosted environment) are stripped, as in reviews, since each of their steps reads input the request doesn't show. Tools the agent runs itself stay.
84+- The request is sent as Diffinery read it, written out again, and OpenAI's `openai-organization` and `openai-project` headers are dropped: the key's own defaults apply.
85+- A refused request costs nothing, and the task goes on.
86+
87+## Other policies
88+
89+| Policy | Default | Checked |
90+| --- | --- | --- |
91+| **Allowed runtimes**: Claude Code, Codex, custom command | All | When a task, review, coordinator or terminal task starts, and when review or coordinator settings are saved. |
92+| **Allowed providers**: Anthropic, OpenAI, OpenRouter | All | As above, and on every model request. A custom command gets only the allowed providers' keys. A terminal's task counts as its runtime's provider: Claude Code as Anthropic, Codex as OpenAI. |
93+| **Most tasks at once per repository**, 1 to 10 | 10 | Each project's own "tasks at once" is held to it. More tasks wait in the queue. The project's settings page says when the organization's limit applies. |
94+| **Allow web terminals** | On | When a terminal opens. Turning it off closes no open terminal, but no new ones open. |
95+| **Let terminals run agent tasks** | On | When a terminal opens for tasks, and when a task is given to one. Under **Only the organization's keys**, only members can run tasks in a terminal, though the person's own subscription pays for them. |
96+| **Allow agent review** | On | When review settings are saved and when each review starts. Turning it off stops new reviews; the pull request's check says why. |
97+
98+Saving the policy is recorded in the audit log as `org.agent_policy.update`, with the new values, and `site_admin: true` when a site administrator saved it. A repository transferred into the organization takes its limit on tasks at once, and one transferred out drops it.
99+
100+## Usage
101+
102+The **Paid with the organization's keys** box on the Settings tab shows owners:
103+
104+- each person's tokens on the organization's keys this month, against the allowance;
105+- the runs the organization's keys paid for in the last 30 days, per repository, person and kind.
106+
107+A person's own **Usage in the last 30 days** (**Settings → Model access**) shows only what they paid for, with their own keys or in their terminal.
108+
109+## REST API
110+
111+| Endpoint | Does | Who |
112+| --- | --- | --- |
113+| `GET /orgs/{org}/diffinery/agent-policy` | The policy | Owners; any token not limited to repositories |
114+| `PUT /orgs/{org}/diffinery/agent-policy` | Change it. Fields left out stay as they are. | Owners, browser session only |
115+| `GET /orgs/{org}/diffinery/model-keys` | The organization's keys: provider, last four characters, who accepted the terms and when | Owners; any token not limited to repositories |
116+| `PUT /orgs/{org}/diffinery/model-keys/{provider}` | Save a key: `{ "key": "sk-ant-...", "accept_terms": true }` | Owners, browser session with a recent sign-in |
117+| `DELETE /orgs/{org}/diffinery/model-keys/{provider}` | Delete a key | Owners, browser session with a recent sign-in |
118+| `GET /orgs/{org}/diffinery/model-usage?days=30` | The runs its keys paid for (1 to 90 days), and each person's tokens this month | Owners; any token not limited to repositories |
119+
120+A token limited to some repositories gets `403` on all of them. Anyone who isn't a member gets `404`, and a member who isn't an owner gets `403 organization owners only`.
121+
122+```json
123+{
124+ "keys": "either",
125+ "runtimes": ["claude-code", "codex", "custom"],
126+ "providers": ["anthropic", "openai", "openrouter"],
127+ "max_running": 4,
128+ "terminals": true,
129+ "terminal_tasks": true,
130+ "review": true,
131+ "member_monthly_tokens": 2000000,
132+ "updated_by": "ayse",
133+ "updated_at": "2026-10-14T07:02:11.000Z"
134+}
135+```
136+
137+`keys` is `personal`, `either` or `organization`. `member_monthly_tokens` is `null` for no limit.
138+
139+## Deleting the organization
140+
141+Deleting an organization removes its model keys, its policy, its allowance counts and the usage its keys paid for (see [Organizations](/docs/accounts/organizations)).
142+
143+Deleting a person's account keeps the runs the organization's keys paid for in its owners' report, under **Deleted account**.
modifiedapps/diffinery/docs/agents/projects.md+2−0View file
@@ -99,6 +99,8 @@
9999
100100 How many tasks run at the same time in the repository, from 1 to 10. The default is 3. A task started beyond it is **Queued**: its page says so, and it starts, oldest first, when a running task finishes or stops, or when an admin raises the number. Follow-ups and retries queue the same way. Agent reviews don't count toward the number and never wait.
101101
102+In an organization's repository, the organization's [agent policy](/docs/agents/organizations#other-policies) can set a lower limit for all its repositories. The lower of the two applies, and this page says so.
103+
102104 ## REST API
103105
104106 | Endpoint | What it does | Who |
modifiedapps/diffinery/docs/agents/review.md+4−4View file
@@ -115,9 +115,9 @@
115115
116116 Diffinery counts tokens itself, as requests pass through the model proxy:
117117
118-- Before a request goes to the provider, the proxy books the most it may cost: a token per 3 bytes of request, plus the output it allows (its `max_tokens` or the endpoint's equivalent, or 32,000 when it sets none; a Codex compaction, which can't be capped, books 128,000). If that could take the review past its budget, the request is refused and the review stops. So requests sent at once can't overrun the budget together.
119-- Once the response has streamed through, the booking is replaced by the usage the provider reports: input, output and cache writes in full, cache reads as a tenth, since providers bill them at about a tenth. A response that reports no usage, or whose usage can't be read, stays charged its booking. An error response costs nothing.
120-- A review can't start background responses, use `previous_response_id`, `conversation` or `prompt` (they pull in context stored at the provider that the booking can't see), ask for more than one answer (`n`), or set routing and paid extras (`models`, `route`, `provider`, `plugins`, `web_search_options`, `speed`, `best_of`). A review through OpenRouter asks only for its configured model. A review's requests keep only tools the agent runs itself (function and custom tools, and client-run ones such as bash, text editor, memory, local shell and apply patch); every tool that runs on the provider is stripped, along with MCP servers, so web search, web fetch, code execution, file search, image generation and any server tool a provider adds later never reach it. Claude Code reviews also run with WebSearch and WebFetch turned off, and Codex reviews with web search disabled. A review's request is sent with every output cap set to the one booked (the largest it set, or 32,000 when it set none), so its booking holds.
118+- Before a request goes to the provider, the proxy books the most it may cost: a token per 3 bytes of request, plus the output it allows (its `max_tokens` or the endpoint's equivalent, or 32,000 when it sets none; a Codex compaction, which can't be capped, books 128,000). If that could take the review past its budget, the request is refused and the review stops. A review has at most 4 requests on their way at once (a fifth gets `429` and is tried again), so requests sent at once can overrun its budget by at most that many: each one's output is capped at what it booked, and only its input can cost more than its estimate (dense text, or images and PDFs, which bill by pixels and pages).
119+- Once the response has streamed through, the booking is replaced by the usage the provider reports: input, output and cache writes in full, cache reads as a tenth, since providers bill them at about a tenth. A response that reports no usage, or whose usage can't be read (a streamed line too long to keep, for example), is charged a token per byte sent, plus its output cap. An error response costs nothing.
120+- A review can't start background responses, pull in input its request doesn't carry (`previous_response_id`, `conversation`, `prompt` or `item_reference` inputs, an Anthropic `container`, or images, documents and files given by URL or file id), send a body that isn't a JSON object, ask for more than one answer (`n`), use an OpenRouter web search model (`:online`) or preset, read back stored responses, or set routing and paid extras (`models`, `route`, `provider`, `plugins`, `web_search_options`, `speed`, `best_of`, `preset`). A review through OpenRouter asks only for its configured model. A review's requests keep only tools the agent runs itself (function and custom tools, and client-run ones such as bash, text editor, memory, a local shell and apply patch); every tool that runs on the provider is stripped, along with MCP servers, so web search, web fetch, code execution, file search, image generation and any server tool a provider adds later never reach it. Claude Code reviews also run with WebSearch and WebFetch turned off, and Codex reviews with web search disabled. A review's request is sent with every output cap set to the one booked (the largest it set, or 32,000 when it set none), so its booking holds.
121121
122122 Every run's model requests and tokens show in its transcript and in the API's `runs` (`model_requests`, `tokens`).
123123
@@ -125,7 +125,7 @@
125125
126126 ## Who pays, and the risks
127127
128-- Every review spends the model key of the admin who turned reviews on. Use a **dedicated key with a spending limit** with your provider as well: Diffinery's [limits](#limits) count reviews, requests and tokens, not money.
128+- Every review spends the model key of the admin who turned reviews on. In an organization's repository, its [agent policy](/docs/agents/organizations#whose-key-pays) can have the organization's key pay instead, while that admin is a member; it can also turn agent review off for all its repositories. Use a **dedicated key with a spending limit** with your provider as well: Diffinery's [limits](#limits) count reviews, requests and tokens, not money.
129129 - While a review runs, anything in its sandbox, including code from the pull request, can use that key through the model proxy. The proxy forwards only the model endpoints, and the review's model token stops working when the agent exits (see [How your key is protected](/docs/agents/tasks#how-your-key-is-protected)).
130130 - The sandbox can't push or write repository content. Its credential acts as the agent with read access.
131131
modifiedapps/diffinery/docs/agents/tasks.md+5−4View file
@@ -127,7 +127,7 @@
127127 - Keys are encrypted at rest and **write-only**: no page, API call or MCP tool returns one. You see the last four characters.
128128 - Only you, signed in in a browser, can save, replace or delete your keys. No token, agent, job or sandbox can.
129129 - Saving a key sends you a security email ("Your Anthropic API key was added to Diffinery"), which you can't turn off.
130-- Keys belong to people. Agents never have model keys of their own, and organization keys aren't available yet.
130+- Keys belong to people. Agents never have model keys of their own. An organization's owners can give it keys of its own, and decide whose key pays in its repositories: see [Agents in organizations](/docs/agents/organizations).
131131 - Subscriptions (Claude Pro or Max, ChatGPT plans) can't be used for headless tasks: providers allow automated use with API keys. You can hand a task to the CLI signed in to [your own web terminal](/docs/agents/terminal#give-it-a-task) instead.
132132
133133 ## Usage and cost
@@ -138,7 +138,7 @@
138138 - **In your terminal**, on a subscription, no request passes Diffinery, so the run shows what the CLI reported: Claude Code at the end of its run, Codex after each turn (`"source": "runtime"`).
139139 - **Cost** is only ever Claude Code's own figure (`total_cost_usd`), at API prices. Diffinery keeps no price list. Codex and custom commands show tokens without a cost, and a subscription covers a terminal run's cost. Your provider's console shows what you were actually billed.
140140
141-**Settings → Model access** shows **Usage in the last 30 days** for every run you pay for: your tasks and your coordinator's, and [agent reviews](/docs/agents/review) in repositories where you turned them on. It is broken down by repository and kind, with each repository named as it was when its runs ended. A repository renamed since shows a line for each name. `GET /api/v3/diffinery/model-usage?days=30` returns the same (1 to 90 days); a token limited to some repositories sees only those.
141+**Settings → Model access** shows **Usage in the last 30 days** for every run you pay for: your tasks and your coordinator's, and [agent reviews](/docs/agents/review) in repositories where you turned them on. Runs an organization's key paid for are on its Settings tab instead, for its owners (see [Agents in organizations](/docs/agents/organizations#usage)). It is broken down by repository and kind, with each repository named as it was when its runs ended. A repository renamed since shows a line for each name. `GET /api/v3/diffinery/model-usage?days=30` returns the same (1 to 90 days); a token limited to some repositories sees only those.
142142
143143 Model requests are limited to **600 a minute per paying person**, across all their runs, on top of each run's own limits. Past that, the proxy answers `429` with `retry-after: 60`, and Claude Code and Codex wait and retry. This caps how fast a runaway task, or a leaked run token, can spend your key. Like the other [rate limits](/docs/reference/rest-api#rate-limits), it is counted in each Cloudflare location, so runs in several locations can go past 600 a minute in total; each run's own limits and budgets still apply.
144144
@@ -147,7 +147,7 @@
147147 Your key never enters a sandbox:
148148
149149 1. When a run's agent starts, Diffinery creates a **model token for that run** (`dmt_...`) and gives it to the runtime's process only, as its API key, with the provider's base URL pointed at Diffinery's model proxy (`https://diffinery.com/-/model/<provider>`).
150-2. The proxy accepts the token only while that run's agent is running. On every request it re-checks that you can still run the task, then forwards the request to the provider with your key added.
150+2. The proxy accepts the token only while that run's agent is running. On every request it re-checks that you can still run the task, and that whoever pays still may (see [Whose key pays](/docs/agents/organizations#whose-key-pays)). Then it forwards the request to the provider with the key added, decrypted for that request only.
151151 3. The proxy forwards only the model endpoints the runtimes use, and refuses everything else with `403`, so an agent can't reach the rest of your provider account (stored files, batches, assistants):
152152
153153 | Provider | Endpoints |
@@ -208,6 +208,7 @@
208208 "runtime": "claude-code",
209209 "agent": { "login": "ayse-bot", "type": "Bot" },
210210 "started_by": { "login": "ayse", "type": "User" },
211+ "paid_by": "person",
211212 "plan_first": false,
212213 "attempts": 1,
213214 "fix_checks": true,
@@ -224,7 +225,7 @@
224225 }
225226 ```
226227
227-A run's `model_requests` and `tokens` are what its agent used through the model proxy, counted from the usage the provider reports (cache reads count as a tenth). An answer that reports no usage counts an estimate: a token per 3 bytes of request, plus the output it allows. `usage` breaks a run's tokens down, and says who counted them (`source`): `proxy`, or `runtime` for a run in your terminal. It is `null` for a run that used none, and the task's `usage` adds up its runs. See [Usage and cost](#usage-and-cost).
228+A run's `model_requests` and `tokens` are what its agent used through the model proxy, counted from the usage the provider reports (cache reads count as a tenth). An answer that reports no usage counts an estimate: a token per 3 bytes of request, plus the output it allows. `usage` breaks a run's tokens down, and says who counted them (`source`): `proxy`, or `runtime` for a run in your terminal. It is `null` for a run that used none, and the task's `usage` adds up its runs. See [Usage and cost](#usage-and-cost). `paid_by` is `person` or `organization`: whose key the task's model requests use, fixed when it started (see [Agents in organizations](/docs/agents/organizations#whose-key-pays)).
228229
229230 Transcript lines have `seq`, `run`, `at`, `stream` (`out` and `err` for the runtime's output, `setup` for installing and pushing, `note` for Diffinery's own lines) and `text`. Claude Code writes `stream-json` lines and Codex writes `--json` lines to `out`.
230231
modifiedapps/diffinery/docs/agents/terminal.md+2−0View file
@@ -15,6 +15,8 @@
1515 2. **An agent** you manage, with write access to the repository, as for [tasks](/docs/agents/tasks#before-you-start).
1616 3. **A browser session.** Terminals open only from a signed-in browser. Tokens, the REST API and MCP can't open one or connect to one.
1717
18+In an organization's repository, its [agent policy](/docs/agents/organizations#other-policies) can turn terminals off, or stop them from running agent tasks.
19+
1820 ## Open a terminal
1921
2022 1. Open the repository's **Project** tab and click **Terminals** (the **Agent tasks** page has the same button), or go to `/<owner>/<repo>/terminals`.
modifiedapps/diffinery/docs/nav.json+1−0View file
@@ -78,6 +78,7 @@
7878 "agents/mcp",
7979 "agents/sandboxes",
8080 "agents/terminal",
81+ "agents/organizations",
8182 "agents/docs-for-agents"
8283 ]
8384 },
modifiedapps/diffinery/docs/reference/limits.md+4−0View file
@@ -278,6 +278,10 @@
278278 | Project conversation post | 20,000 characters | ``422 `body` is longer than 20000 characters``. The conversation API and `project_posts` return at most 100 posts per call. |
279279 | Model requests per person | 600 a minute, across all the runs you pay for | The model proxy answers `429` with `retry-after: 60`; the run goes on, and Claude Code and Codex retry. Counted in each Cloudflare location, like the [rate limits](/docs/reference/rest-api#rate-limits), so it isn't a global total. |
280280 | Usage report | 1 to 90 days, 200 lines | `GET /diffinery/model-usage` and **Settings → Model access**. See [Usage and cost](/docs/agents/tasks#usage-and-cost). |
281+| An organization's tasks at once per repository | 1 to 10, default 10 | Set in its [agent policy](/docs/agents/organizations#other-policies); each project's own setting is held to it. |
282+| Tokens per person on an organization's keys | 10,000 to 10 billion a calendar month (UTC), no limit by default | The model proxy refuses a request that could go over with `403`, and the task stops. See [Monthly allowance per person](/docs/agents/organizations#monthly-allowance-per-person). |
283+| Model requests at once | 4 per review or budgeted run, and 4 per person on an organization's keys while it sets an allowance | The model proxy answers `429` with `retry-after: 5`, and the CLI tries again. A request still streaming after 15 minutes stops counting. |
284+| An organization's usage report | 1 to 90 days, 500 lines | `GET /orgs/{org}/diffinery/model-usage` and the organization's **Settings** tab. |
281285
282286 See [Agent tasks](/docs/agents/tasks). For the Today view's limits (repositories active in the last 30 days, at most 100 of them, 50 open pull requests each), see [Today](/docs/agents/today).
283287
addedapps/diffinery/migrations/0031_org_agent_policies.sql+41−0View file
@@ -1,0 +1,41 @@
1+-- Phase 5b: organizations' agent policies and model keys.
2+--
3+-- An organization's model keys live in `model_keys` under the organization's id, like a person's under theirs;
4+-- `opted_in_by` is the owner who accepted the provider's terms for the organization (null for personal keys).
5+ALTER TABLE model_keys ADD COLUMN opted_in_by TEXT;
6+
7+-- One JSON row per organization that set a policy: whose key pays, allowed runtimes and providers, most tasks
8+-- at once per repository, terminals, agent review, and the monthly token allowance per person on its keys.
9+CREATE TABLE org_agent_policies (
10+ org_id TEXT PRIMARY KEY,
11+ json TEXT NOT NULL,
12+ updated_by TEXT NOT NULL,
13+ updated_at TEXT NOT NULL
14+);
15+
16+-- Tokens each person spent on an organization's keys per calendar month (UTC, `2026-10`), weighted as budgets
17+-- weigh them, for the policy's monthly allowance. The model proxy books each request's estimate here before it
18+-- goes upstream, and settles it to what the provider reported.
19+CREATE TABLE org_key_spend (
20+ org_id TEXT NOT NULL,
21+ person_id TEXT NOT NULL,
22+ month TEXT NOT NULL,
23+ tokens INTEGER NOT NULL,
24+ PRIMARY KEY (org_id, person_id, month)
25+);
26+
27+-- Whose key paid for each run: the person's id, or the organization's when its key paid. Null for runs before 5b
28+-- and for runs on a subscription in a terminal, which are the person's own.
29+ALTER TABLE agent_usage ADD COLUMN key_owner TEXT;
30+CREATE INDEX agent_usage_key_owner ON agent_usage(key_owner, ended_at);
31+
32+-- Model requests in flight that a limit counts (the model proxy's leases): a review's or budgeted run's
33+-- (`run:<repo>:<task>:<run>`), and a person's on an organization's keys while it sets an allowance
34+-- (`org:<org>:<person>`). Each row is one request, deleted when its answer is settled; a lease that outlives
35+-- that (a Worker that died mid-request) stops counting when it expires.
36+CREATE TABLE model_in_flight (
37+ id TEXT PRIMARY KEY,
38+ scope TEXT NOT NULL,
39+ expires_at TEXT NOT NULL
40+);
41+CREATE INDEX model_in_flight_scope ON model_in_flight(scope, expires_at);
modifiedapps/diffinery/src/accounts/lifecycle.ts+3−1View file
@@ -106,7 +106,9 @@
106106 forMe("DELETE FROM account_links WHERE account_id = ?"),
107107 forIds("DELETE FROM model_keys WHERE account_id IN (?)"),
108108 forMe("DELETE FROM terminal_terms WHERE account_id = ?"),
109- forMe("DELETE FROM agent_usage WHERE person_id = ?"),
109+ // Runs an organization's key paid for stay in its owners' report, as a deleted account.
110+ db.prepare("DELETE FROM agent_usage WHERE person_id = ? AND (key_owner IS NULL OR key_owner = ?)").bind(account.id, account.id),
111+ forMe("DELETE FROM org_key_spend WHERE person_id = ?"),
110112 forIds("DELETE FROM account_quotas WHERE account_id IN (?)"),
111113 forIds("DELETE FROM compute_usage WHERE account_id IN (?)"),
112114 forIds("DELETE FROM login_redirects WHERE account_id IN (?)"),
modifiedapps/diffinery/src/agentic/ProjectDO.ts+50−18View file
@@ -100,6 +100,8 @@
100100 check_fix_sha: string | null;
101101 /** Land (phase 3e): turn on auto-merge for its pull request, which merges once a person approves and checks pass. */
102102 land: number;
103+ /** Phase 5b: whose key pays, the person's id or an organization's (org-policy.ts); null before 5b, meaning the person's. */
104+ key_owner: string | null;
103105 created_at: string;
104106 updated_at: string;
105107 };
@@ -307,6 +309,7 @@
307309 ["tasks", "check_fix_sha", "check_fix_sha TEXT"],
308310 ["tasks", "land", "land INTEGER NOT NULL DEFAULT 0"],
309311 ["runs", "terminal_id", "terminal_id TEXT"],
312+ ["tasks", "key_owner", "key_owner TEXT"],
310313 ["runs", "reported_input", "reported_input INTEGER NOT NULL DEFAULT 0"],
311314 ["runs", "reported_output", "reported_output INTEGER NOT NULL DEFAULT 0"],
312315 ["runs", "reported_cache_read", "reported_cache_read INTEGER NOT NULL DEFAULT 0"],
@@ -381,6 +384,10 @@
381384 land?: boolean;
382385 /** Run it in the person's web terminal (phase 4b; agentic/terminals.ts checked it is theirs and open). */
383386 terminal?: InTerminal;
387+ /** Phase 5b: whose key pays (org-policy.ts payerFor); the person's when left out. */
388+ keyOwner?: string;
389+ /** Phase 5b: the owning organization's limit on tasks at once, refreshed by each start (null: none). */
390+ orgMaxRunning?: number | null;
384391 }
385392
386393 /** A web terminal a run goes to: its id and its sandbox. */
@@ -397,6 +404,8 @@
397404 headSha: string;
398405 settings: ReviewSettings;
399406 providers: Provider[];
407+ /** Phase 5b: whose key pays for this review. */
408+ keyOwner?: string;
400409 /** Who asked; null for an automatic review. */
401410 requestedBy: string | null;
402411 }
@@ -439,9 +448,23 @@
439448 projectSettings(): ProjectSettings {
440449 const kv = new Map(this.sql.exec<{ k: string; v: string }>("SELECT k, v FROM project_settings").toArray().map((r) => [r.k, r.v]));
441450 const planFirst = kv.get("plan_first");
442- return { maxRunning: Number(kv.get("max_running") ?? MAX_RUNNING_DEFAULT) || MAX_RUNNING_DEFAULT, instructions: kv.get("instructions") ?? "", planFirst: isPlanFirst(planFirst) ? planFirst : "off" };
451+ const org = kv.get("org_max_running");
452+ return { maxRunning: Number(kv.get("max_running") ?? MAX_RUNNING_DEFAULT) || MAX_RUNNING_DEFAULT, orgMaxRunning: org ? Number(org) : null, instructions: kv.get("instructions") ?? "", planFirst: isPlanFirst(planFirst) ? planFirst : "off" };
443453 }
444454
455+ /** Tasks at once here: the project's own setting, clamped to its organization's policy (phase 5b). */
456+ private maxRunning(): number {
457+ const s = this.projectSettings();
458+ return s.orgMaxRunning === null ? s.maxRunning : Math.min(s.maxRunning, s.orgMaxRunning);
459+ }
460+
461+ /** Phase 5b: the owning organization's limit on tasks at once (null: none), set when its policy is saved and at each task start. */
462+ setOrgMaxRunning(n: number | null): void {
463+ if (n === null) this.sql.exec("DELETE FROM project_settings WHERE k = 'org_max_running'");
464+ else this.sql.exec("INSERT INTO project_settings (k, v) VALUES ('org_max_running', ?) ON CONFLICT (k) DO UPDATE SET v = excluded.v", String(n));
465+ this.promote();
466+ }
467+
445468 setProjectSettings(s: Partial<ProjectSettings>): ProjectSettings {
446469 if (s.maxRunning !== undefined) this.sql.exec("INSERT INTO project_settings (k, v) VALUES ('max_running', ?) ON CONFLICT (k) DO UPDATE SET v = excluded.v", String(s.maxRunning));
447470 if (s.instructions !== undefined) this.sql.exec("INSERT INTO project_settings (k, v) VALUES ('instructions', ?) ON CONFLICT (k) DO UPDATE SET v = excluded.v", s.instructions);
@@ -514,7 +537,7 @@
514537 * task per person, a run per post. Posts that arrive while it runs wait and
515538 * go together in its next run.
516539 */
517- async handlePost(n: number, settings: CoordinatorSettings, providers: Provider[], repo: { id: string; base: string }): Promise<{ task: number; status: "started" | "queued" }> {
540+ async handlePost(n: number, settings: CoordinatorSettings, providers: Provider[], repo: { id: string; base: string }, keyOwner?: string): Promise<{ task: number; status: "started" | "queued" }> {
518541 const p = this.post1(n);
519542 if (!p || p.kind !== "person") throw new HttpError(404, "no such post");
520543 if (p.handled_by) throw new HttpError(409, `post #${n} is already being handled`);
@@ -543,7 +566,7 @@
543566 t = this.mustTask(number);
544567 }
545568 // The person's current settings decide how it runs.
546- this.sql.exec("UPDATE tasks SET runtime = ?, command = ?, via = ?, model = ?, agent_id = ?, agent_login = ?, base = ?, branch = ? WHERE number = ?", settings.runtime, settings.command, settings.via, settings.model, settings.agent.id, settings.agent.login, repo.base, repo.base, t.number);
569+ this.sql.exec("UPDATE tasks SET runtime = ?, command = ?, via = ?, model = ?, agent_id = ?, agent_login = ?, base = ?, branch = ?, key_owner = ? WHERE number = ?", settings.runtime, settings.command, settings.via, settings.model, settings.agent.id, settings.agent.login, repo.base, repo.base, keyOwner ?? settings.person.id, t.number);
547570 this.sql.exec("UPDATE posts SET handled_by = ? WHERE n = ?", t.number, n);
548571 t = this.mustTask(t.number);
549572 if (this.activeRun(t.number)) {
@@ -564,7 +587,7 @@
564587 const branch = input.pull?.head ?? taskBranch(number, input.title || input.goal);
565588 const title = input.title ? taskTitle(input.title) : taskTitle(input.goal);
566589 this.sql.exec(
567- "INSERT INTO tasks (number, id, repo_id, goal, title, runtime, command, via, model, agent_id, agent_login, person_id, person_login, base, branch, state, pr_number, autofix_of, coordinator_post, token_budget, after_tasks, plan_first, plan_state, parent_task, attempts, fix_checks, land, created_at, updated_at) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 'running', ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",
590+ "INSERT INTO tasks (number, id, repo_id, goal, title, runtime, command, via, model, agent_id, agent_login, person_id, person_login, base, branch, state, pr_number, autofix_of, coordinator_post, token_budget, after_tasks, plan_first, plan_state, parent_task, attempts, fix_checks, land, key_owner, created_at, updated_at) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 'running', ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",
568591 number,
569592 crypto.randomUUID(),
570593 input.repoId,
@@ -591,9 +614,11 @@
591614 Math.min(Math.max(input.attempts ?? 1, 1), MAX_ATTEMPTS),
592615 input.fixChecks ? 1 : 0,
593616 input.land ? 1 : 0,
617+ input.keyOwner ?? input.person.id,
594618 at,
595619 at,
596620 );
621+ if (input.orgMaxRunning !== undefined) this.setOrgMaxRunning(input.orgMaxRunning);
597622 this.event(number, `${input.autofixOf ? "Autofix" : input.coordinator ? `${input.person.login}'s coordinator` : input.person.login} started task #${number}: ${title}`);
598623 if (input.terminal) this.startRun(number, "start", null, [], null, input.terminal);
599624 else if (input.planFirst) this.startRun(number, "plan", null, input.providers);
@@ -708,13 +733,13 @@
708733 * Split a waiting plan into tasks: the person's coordinator gets it as a post of theirs, and the tasks it starts
709734 * link back to this one, which is then done with.
710735 */
711- async splitPlan(number: number, settings: CoordinatorSettings, providers: Provider[], repo: { id: string; base: string }): Promise<{ post: number; task: number; status: "started" | "queued" }> {
736+ async splitPlan(number: number, settings: CoordinatorSettings, providers: Provider[], repo: { id: string; base: string }, keyOwner?: string): Promise<{ post: number; task: number; status: "started" | "queued" }> {
712737 const t = this.waitingPlan(number);
713738 const head = `Split the plan of task #${number} (${t.title}) into tasks. Its goal was:\n\n${t.goal}\n\nThe plan:\n\n`;
714739 const p = this.addPost("person", settings.person, `${head}${t.plan!}`.slice(0, MAX_POST), number);
715740 this.sql.exec("UPDATE posts SET split_of = ? WHERE n = ?", number, p.n);
716741 this.sql.exec("UPDATE tasks SET plan_state = 'split', updated_at = ? WHERE number = ?", now(), number);
717- const out = await this.handlePost(p.n, settings, providers, repo);
742+ const out = await this.handlePost(p.n, settings, providers, repo, keyOwner);
718743 await this.index(number);
719744 return { post: p.n, ...out };
720745 }
@@ -948,7 +973,7 @@
948973 t.number,
949974 );
950975 }
951- this.sql.exec("UPDATE tasks SET via = ?, model = ? WHERE number = ?", s.via ?? null, s.model ?? null, t.number);
976+ this.sql.exec("UPDATE tasks SET via = ?, model = ?, key_owner = ? WHERE number = ?", s.via ?? null, s.model ?? null, input.keyOwner ?? s.person.id, t.number);
952977 t = this.mustTask(t.number);
953978 if (this.activeRun(t.number)) {
954979 // One more review after this one covers every push in between.
@@ -999,15 +1024,17 @@
9991024 async modelAuth(
10001025 token: string,
10011026 req: { provider: Provider; estimate: Usage | null; reviewRefuse?: string; model?: string | null; routing?: string },
1002- ): Promise<{ task: number; run: number; personId: string; agentId: string; repoId: string; bounded: boolean } | { limit: string } | { refuse: string } | { busy: string } | null> {
1027+ ): Promise<{ task: number; run: number; personId: string; keyOwner: string; agentId: string; repoId: string; bounded: boolean; orgKey: boolean } | { limit: string } | { refuse: string } | { busy: string } | null> {
10031028 const hash = await sha256(token);
10041029 const r = this.sql.exec<RunRow>("SELECT * FROM runs WHERE token_hash = ? AND step = 'agent'", hash).toArray()[0];
10051030 if (!r) return null;
10061031 const t = this.task(r.task)!;
10071032 if (!(JSON.parse(r.providers) as Provider[]).includes(req.provider)) return { refuse: `this task doesn't use ${PROVIDERS[req.provider].name}` };
10081033 const bound = this.bound(t);
1009- if (bound && req.reviewRefuse) return { refuse: req.reviewRefuse };
1010- const pinned = pinRefusal(req, { review: !!bound, model: t.model });
1034+ // A run on an organization's key is held to its bookings too, so the monthly allowance per person holds (phase 5b).
1035+ const orgKey = !!t.key_owner && t.key_owner !== t.person_id;
1036+ if ((bound || orgKey) && req.reviewRefuse) return { refuse: req.reviewRefuse };
1037+ const pinned = pinRefusal(req, { review: !!bound, model: t.model, orgKey });
10111038 if (pinned) return { refuse: pinned };
10121039 // Phase 5a: the paying person's rate across all their runs. Nothing is booked or counted, and the run goes on: the CLI retries.
10131040 if (await modelRateLimited(this.env, t.person_id)) return { busy: `Too many model requests from ${t.person_login}'s agents: at most ${MODEL_RATE_LIMIT} a minute. Try again in a minute.` };
@@ -1033,7 +1060,7 @@
10331060 }
10341061 }
10351062 this.sql.exec("UPDATE runs SET model_calls = ?, tokens = tokens + ? WHERE task = ? AND seq = ?", calls, cost, r.task, r.seq);
1036- return { task: t.number, run: r.seq, personId: t.person_id, agentId: t.agent_id, repoId: t.repo_id, bounded: !!bound };
1063+ return { task: t.number, run: r.seq, personId: t.person_id, keyOwner: t.key_owner ?? t.person_id, agentId: t.agent_id, repoId: t.repo_id, bounded: !!bound, orgKey };
10371064 }
10381065
10391066 /**
@@ -1052,7 +1079,9 @@
10521079 * none), and replaces the `booked` estimate. A request whose usage can't be read is never settled, and so
10531080 * stays charged its estimate.
10541081 */
1055- async chargeUsage(task: number, run: number, used: Usage, booked: Usage | null): Promise<void> {
1082+ async chargeUsage(task: number, run: number, used: Usage, booked: Usage | null, unsent = false): Promise<void> {
1083+ // A request the proxy refused before sending it (phase 5b: whoever pays may no longer) isn't one of the run's requests.
1084+ if (unsent) this.sql.exec("UPDATE runs SET model_calls = MAX(model_calls - 1, 0) WHERE task = ? AND seq = ?", task, run);
10561085 this.sql.exec(
10571086 "UPDATE runs SET tokens = MAX(tokens + ?, 0), input_tokens = input_tokens + ?, output_tokens = output_tokens + ?, cache_read_tokens = cache_read_tokens + ?, cache_write_tokens = cache_write_tokens + ? WHERE task = ? AND seq = ?",
10581087 weighted(used) - (booked ? weighted(booked) : 0),
@@ -1137,8 +1166,9 @@
11371166 const waits = this.waitsFor(t);
11381167 if (waits.length) this.note(number, seq, `Queued: it starts when ${waits.map((n) => `task #${n}`).join(" and ")} ${waits.length === 1 ? "is" : "are"} ready.`);
11391168 else {
1140- const { maxRunning } = this.projectSettings();
1141- this.note(number, seq, `Queued: ${maxRunning} ${maxRunning === 1 ? "task is" : "tasks are"} already running, the project's limit. It starts when one finishes.`);
1169+ const maxRunning = this.maxRunning();
1170+ const org = this.projectSettings().orgMaxRunning === maxRunning && maxRunning < this.projectSettings().maxRunning;
1171+ this.note(number, seq, `Queued: ${maxRunning} ${maxRunning === 1 ? "task is" : "tasks are"} already running, the ${org ? "organization's" : "project's"} limit. It starts when one finishes.`);
11421172 }
11431173 }
11441174 this.arm(0);
@@ -1147,7 +1177,7 @@
11471177 /** Start queued runs, oldest first, while the project has room; returns the tasks started. */
11481178 private promote(): number[] {
11491179 const running = this.sql.exec<{ n: number }>(`SELECT count(*) AS n FROM runs r JOIN tasks t ON t.number = r.task WHERE t.kind = 'task' AND r.terminal_id IS NULL AND r.step IN (${RUNNING.map(() => "?").join(", ")})`, ...RUNNING).one().n;
1150- let room = this.projectSettings().maxRunning - running;
1180+ let room = this.maxRunning() - running;
11511181 const queued: RunRow[] = [];
11521182 const at = now();
11531183 for (const r of this.sql.exec<RunRow>("SELECT * FROM runs WHERE step = 'queued' ORDER BY started_at, task, seq").toArray()) {
@@ -1544,6 +1574,8 @@
15441574 base: x.base ?? t.base,
15451575 providers,
15461576 coordinator: { post: post!, tokenBudget: settings.tokensPerTask, after: x.after.map((i) => started[i]!), parent: splitOf },
1577+ // The tasks it starts are paid like its own run.
1578+ keyOwner: t.key_owner ?? t.person_id,
15471579 planFirst: this.projectSettings().planFirst === "required",
15481580 fixChecks: true,
15491581 });
@@ -1718,7 +1750,7 @@
17181750 else if (this.dailyRuns() >= repoLimit) this.note(t.number, r.seq, `Autofix didn't start: the repository reached its daily limit of ${repoLimit} reviews and autofix runs.`);
17191751 else {
17201752 this.sql.exec("UPDATE tasks SET autofixes = autofixes + 1 WHERE number = ?", t.number);
1721- const fix = await this.create({ repoId: t.repo_id, goal: fixGoal(pr, blocking), runtime: settings.runtime, command: settings.command, route: { via: settings.via ?? null, model: settings.model ?? null }, agent: settings.agent, person: settings.person, base: issue.pull.base, providers: JSON.parse(r.providers) as Provider[], pull: { number: pr, head: issue.pull.head }, autofixOf: t.number });
1753+ const fix = await this.create({ repoId: t.repo_id, goal: fixGoal(pr, blocking), runtime: settings.runtime, command: settings.command, route: { via: settings.via ?? null, model: settings.model ?? null }, agent: settings.agent, person: settings.person, base: issue.pull.base, providers: JSON.parse(r.providers) as Provider[], pull: { number: pr, head: issue.pull.head }, autofixOf: t.number, keyOwner: t.key_owner ?? t.person_id });
17221754 this.note(t.number, r.seq, `Autofix started task #${fix.number} for the blocking ${blocking.length === 1 ? "finding" : "findings"}.`);
17231755 }
17241756 }
@@ -1994,9 +2026,9 @@
19942026 // The repository's name as of now, kept on the row: a later rename or transfer never shows to someone who lost access (Security, 5a).
19952027 const repo = await new Directory(this.env).repoById(t.repo_id);
19962028 await this.env.DB.prepare(
1997- "INSERT INTO agent_usage (repo_id, repository, task, run, person_id, kind, runtime, providers, source, requests, input_tokens, output_tokens, cache_read_tokens, cache_write_tokens, cost_usd, ended_at) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) ON CONFLICT (repo_id, task, run) DO UPDATE SET source = excluded.source, requests = excluded.requests, input_tokens = excluded.input_tokens, output_tokens = excluded.output_tokens, cache_read_tokens = excluded.cache_read_tokens, cache_write_tokens = excluded.cache_write_tokens, cost_usd = excluded.cost_usd",
2029+ "INSERT INTO agent_usage (repo_id, repository, task, run, person_id, kind, runtime, providers, source, requests, input_tokens, output_tokens, cache_read_tokens, cache_write_tokens, cost_usd, ended_at, key_owner) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) ON CONFLICT (repo_id, task, run) DO UPDATE SET source = excluded.source, requests = excluded.requests, input_tokens = excluded.input_tokens, output_tokens = excluded.output_tokens, cache_read_tokens = excluded.cache_read_tokens, cache_write_tokens = excluded.cache_write_tokens, cost_usd = excluded.cost_usd",
19982030 )
1999- .bind(t.repo_id, repo ? `${repo.owner_login}/${repo.name}` : "", task, seq, t.person_id, t.kind, t.runtime, providers, u.source, u.requests, u.input, u.output, u.cacheRead, u.cacheWrite, u.costUsd, r.ended_at)
2031+ .bind(t.repo_id, repo ? `${repo.owner_login}/${repo.name}` : "", task, seq, t.person_id, t.kind, t.runtime, providers, u.source, u.requests, u.input, u.output, u.cacheRead, u.cacheWrite, u.costUsd, r.ended_at, r.terminal_id ? null : (t.key_owner ?? t.person_id))
20002032 .run()
20012033 .catch((e) => console.error("agent_usage failed", e));
20022034 }
modifiedapps/diffinery/src/agentic/api.ts+68−3View file
@@ -1,13 +1,14 @@
11 import { Hono } from "hono";
22 import type { Env } from "../env.js";
3-import { requireAccount, type AppContext, type AppVars } from "../auth/principal.js";
3+import { requireAccount, requireRecentSignIn, type AppContext, type AppVars } from "../auth/principal.js";
44 import { loadRepo } from "../http/common.js";
55 import { HttpError } from "../lib/util.js";
66 import { isProvider, ModelKeys, type ModelKeyInfo } from "./keys.js";
77 import { approvePlan, attemptJson, compareAttempts, getTask, pickAttempt, setTaskOptions, lineJson, messageTask, myTasks, projectStub, requirePersonSession, requireTaskReader, retryTask, startTask, stopTask, taskJson, taskNumber, type StartInput } from "./tasks.js";
88 import { askForReview, findingJson, reviewSettings, reviewSettingsJson, saveReviewSettings, type SettingsInput } from "./reviews.js";
99 import { gatherToday, todayJson } from "./today.js";
10-import { usageDays, usageReport, usageReportJson } from "./ledger.js";
10+import { orgUsageReport, orgUsageReportJson, usageDays, usageReport, usageReportJson } from "./ledger.js";
11+import { orgPolicy, orgSpend, parsePolicy, policyJson, savePolicy, spendMonth, type PolicyInput } from "./org-policy.js";
1112 import { boardJson, coordinatorJson, handlePost, splitPlan, loadBoard, memoryJson, postJson, postToProject, saveCoordinator, saveMemory, saveProjectSettings, settingsJson, type CoordinatorInput } from "./projects.js";
1213
1314 async function jsonBody<T>(c: AppContext): Promise<T> {
@@ -35,7 +36,7 @@
3536
3637 // ---------------------------------------------------------------- model access (agentic/keys.ts)
3738
38-const keyJson = (k: ModelKeyInfo) => ({ provider: k.provider, hint: k.hint, opted_in_at: k.optedInAt, created_at: k.createdAt, updated_at: k.updatedAt });
39+const keyJson = (k: ModelKeyInfo) => ({ provider: k.provider, hint: k.hint, opted_in_at: k.optedInAt, ...(k.optedInBy ? { opted_in_by: k.optedInBy } : {}), created_at: k.createdAt, updated_at: k.updatedAt });
3940
4041 agenticApi.get("/diffinery/model-keys", async (c) => c.json((await new ModelKeys(c.env).list(requireAccount(c).id)).map(keyJson)));
4142
@@ -69,6 +70,70 @@
6970 return c.body(null, 204);
7071 });
7172
73+// ---------------------------------------------------------------- organizations' agent policy and keys (phase 5b, org-policy.ts)
74+
75+/**
76+ * An organization's agent settings: its owners only (and site admins, audited as themselves). A token limited to
77+ * repositories can't reach them. Writes need a browser session; key writes also a sign-in from the last 2 hours.
78+ */
79+async function orgOwner(c: AppContext, need: "read" | "write" | "key") {
80+ const me = need === "read" ? requireAccount(c) : requirePersonSession(c);
81+ if (tokenRepoIds(c)) throw new HttpError(403, "a token limited to repositories can't use an organization's agent settings");
82+ const dir = c.get("dir");
83+ const org = await dir.accountByLogin(c.req.param("org") ?? "");
84+ if (!org || org.kind !== "org" || org.status === "disabled") throw new HttpError(404, "not found");
85+ const role = await dir.orgRole(org.id, me.id);
86+ if (role !== "owner" && !me.site_admin) throw new HttpError(role ? 403 : 404, role ? "organization owners only" : "not found");
87+ if (need === "key") requireRecentSignIn(c, `/${org.login}?tab=settings`);
88+ return { me, org, dir, asSiteAdmin: role !== "owner" };
89+}
90+
91+agenticApi.get("/orgs/:org/diffinery/agent-policy", async (c) => {
92+ const { org } = await orgOwner(c, "read");
93+ return c.json(policyJson(await orgPolicy(c.env, org.id)));
94+});
95+
96+agenticApi.put("/orgs/:org/diffinery/agent-policy", async (c) => {
97+ const { me, org, dir } = await orgOwner(c, "write");
98+ const p = parsePolicy(await jsonBody<PolicyInput>(c), await orgPolicy(c.env, org.id));
99+ return c.json(policyJson(await savePolicy(c.env, dir, org, me, p, (x) => c.executionCtx.waitUntil(x))));
100+});
101+
102+agenticApi.get("/orgs/:org/diffinery/model-keys", async (c) => {
103+ const { org } = await orgOwner(c, "read");
104+ return c.json((await new ModelKeys(c.env).list(org.id)).map(keyJson));
105+});
106+
107+agenticApi.put("/orgs/:org/diffinery/model-keys/:provider", async (c) => {
108+ const { me, org, dir, asSiteAdmin } = await orgOwner(c, "key");
109+ const provider = c.req.param("provider");
110+ if (!isProvider(provider)) throw new HttpError(404, "unknown provider: use anthropic, openai or openrouter");
111+ const b = await jsonBody<{ key?: unknown; accept_terms?: unknown }>(c);
112+ if (typeof b.key !== "string") throw new HttpError(422, "`key` is required");
113+ const k = await new ModelKeys(c.env).set(org, provider, b.key, b.accept_terms === true, me);
114+ c.executionCtx.waitUntil(dir.audit({ actorId: me.id, actorLogin: me.login, action: "org.model_key.set", target: org.login, detail: { provider, hint: k.hint, opted_in_at: k.optedInAt, ...(asSiteAdmin ? { site_admin: true } : {}) } }));
115+ return c.json(keyJson(k));
116+});
117+
118+agenticApi.delete("/orgs/:org/diffinery/model-keys/:provider", async (c) => {
119+ const { me, org, dir, asSiteAdmin } = await orgOwner(c, "key");
120+ const provider = c.req.param("provider");
121+ if (!isProvider(provider)) throw new HttpError(404, "unknown provider: use anthropic, openai or openrouter");
122+ if (!(await new ModelKeys(c.env).delete(org.id, provider))) throw new HttpError(404, "no key for that provider");
123+ c.executionCtx.waitUntil(dir.audit({ actorId: me.id, actorLogin: me.login, action: "org.model_key.delete", target: org.login, detail: { provider, ...(asSiteAdmin ? { site_admin: true } : {}) } }));
124+ return c.body(null, 204);
125+});
126+
127+/** The runs the organization's keys paid for over the last `days`, and each person's tokens this month against the allowance. */
128+agenticApi.get("/orgs/:org/diffinery/model-usage", async (c) => {
129+ const { org } = await orgOwner(c, "read");
130+ const [report, policy, spend] = await Promise.all([orgUsageReport(c.env, org.id, usageDays(c.req.query("days"))), orgPolicy(c.env, org.id), orgSpend(c.env, org.id)]);
131+ return c.json({
132+ ...orgUsageReportJson(report),
133+ month: { month: spendMonth(), member_monthly_tokens: policy.memberMonthlyTokens, people: spend.map((x) => ({ person: x.login, tokens: x.tokens })) },
134+ });
135+});
136+
72137 // ---------------------------------------------------------------- agent tasks (agentic/tasks.ts)
73138
74139 const base = (c: AppContext) => c.env.PUBLIC_URL.replace(/\/$/, "");
addedapps/diffinery/src/agentic/inflight.ts+26−0View file
@@ -1,0 +1,26 @@
1+import type { Env } from "../env.js";
2+import { randomToken } from "../lib/util.js";
3+
4+/**
5+ * Model requests at once that a limit counts (phase 5b). A booking bounds a request's output and the input it
6+ * carries, but not what an image or PDF in it bills (by pixels and pages, not bytes), so runs held to their
7+ * bookings are also held to a few requests at once: past a budget or an allowance, at most that many can be on
8+ * their way.
9+ */
10+export const MAX_IN_FLIGHT = 4;
11+/** How long a lease counts at most: an answer still streaming after this stops counting. */
12+export const IN_FLIGHT_LEASE_MS = 15 * 60_000;
13+
14+/** Take one of `scope`'s slots for a request: its lease id, or null when all are taken. One atomic step. */
15+export async function takeSlot(env: Pick<Env, "DB">, scope: string, at = Date.now()): Promise<string | null> {
16+ const id = randomToken(16);
17+ const [, taken] = await env.DB.batch([
18+ env.DB.prepare("DELETE FROM model_in_flight WHERE scope = ? AND expires_at <= ?").bind(scope, new Date(at).toISOString()),
19+ env.DB.prepare("INSERT INTO model_in_flight (id, scope, expires_at) SELECT ?, ?, ? WHERE (SELECT COUNT(*) FROM model_in_flight WHERE scope = ?) < ?").bind(id, scope, new Date(at + IN_FLIGHT_LEASE_MS).toISOString(), scope, MAX_IN_FLIGHT),
20+ ]);
21+ return taken?.meta.changes ? id : null;
22+}
23+
24+export async function freeSlot(env: Pick<Env, "DB">, id: string): Promise<void> {
25+ await env.DB.prepare("DELETE FROM model_in_flight WHERE id = ?").bind(id).run();
26+}
modifiedapps/diffinery/src/agentic/keys.ts+17−10View file
@@ -65,6 +65,8 @@
6565 /** The key's last four characters. */
6666 hint: string;
6767 optedInAt: string;
68+ /** An organization's key (phase 5b): the owner who accepted the provider's terms for it. */
69+ optedInBy: string | null;
6870 createdAt: string;
6971 updatedAt: string;
7072 }
@@ -75,10 +77,10 @@
7577 constructor(private env: Env) {}
7678
7779 async list(accountId: string): Promise<ModelKeyInfo[]> {
78- const r = await this.env.DB.prepare("SELECT provider, hint, opted_in_at, created_at, updated_at FROM model_keys WHERE account_id = ? ORDER BY provider")
80+ const r = await this.env.DB.prepare("SELECT k.provider, k.hint, k.opted_in_at, a.login AS opted_in_by, k.created_at, k.updated_at FROM model_keys k LEFT JOIN accounts a ON a.id = k.opted_in_by WHERE k.account_id = ? ORDER BY k.provider")
7981 .bind(accountId)
80- .all<{ provider: Provider; hint: string; opted_in_at: string; created_at: string; updated_at: string }>();
81- return r.results.map((k) => ({ provider: k.provider, hint: k.hint, optedInAt: k.opted_in_at, createdAt: k.created_at, updatedAt: k.updated_at }));
82+ .all<{ provider: Provider; hint: string; opted_in_at: string; opted_in_by: string | null; created_at: string; updated_at: string }>();
83+ return r.results.map((k) => ({ provider: k.provider, hint: k.hint, optedInAt: k.opted_in_at, optedInBy: k.opted_in_by, createdAt: k.created_at, updatedAt: k.updated_at }));
8284 }
8385
8486 async providers(accountId: string): Promise<Set<Provider>> {
@@ -88,21 +90,26 @@
8890 /**
8991 * Save (or replace) a key. The caller has checked that this is the person
9092 * themselves in a browser session; `optIn` is their acceptance of the notice.
93+ * For an organization's key (phase 5b), `account` is the organization and `by`
94+ * the owner accepting the provider's terms for it; every owner is emailed.
9195 */
92- async set(account: { id: string }, provider: Provider, key: string, optIn: boolean): Promise<ModelKeyInfo> {
93- if (!optIn) throw new HttpError(422, `confirm that you use ${PROVIDERS[provider].name} under your own account and its terms`);
96+ async set(account: { id: string; kind?: string; login?: string }, provider: Provider, key: string, optIn: boolean, by?: { id: string; login: string }): Promise<ModelKeyInfo> {
97+ const org = account.kind === "org";
98+ if (!optIn) throw new HttpError(422, org ? `confirm that ${account.login} uses ${PROVIDERS[provider].name} under its own account and its terms` : `confirm that you use ${PROVIDERS[provider].name} under your own account and its terms`);
9499 const k = key.trim();
95100 if (!PROVIDERS[provider].pattern.test(k)) throw new HttpError(422, `that doesn't look like an ${PROVIDERS[provider].name} API key`);
96101 const at = now();
97102 const ciphertext = await encryptSecret(this.env.MASTER_KEY, scope(account.id, provider), k);
98103 const hint = k.slice(-4);
99104 await this.env.DB.prepare(
100- "INSERT INTO model_keys (account_id, provider, ciphertext, hint, opted_in_at, created_at, updated_at) VALUES (?, ?, ?, ?, ?, ?, ?) ON CONFLICT (account_id, provider) DO UPDATE SET ciphertext = excluded.ciphertext, hint = excluded.hint, opted_in_at = excluded.opted_in_at, updated_at = excluded.updated_at",
105+ "INSERT INTO model_keys (account_id, provider, ciphertext, hint, opted_in_at, opted_in_by, created_at, updated_at) VALUES (?, ?, ?, ?, ?, ?, ?, ?) ON CONFLICT (account_id, provider) DO UPDATE SET ciphertext = excluded.ciphertext, hint = excluded.hint, opted_in_at = excluded.opted_in_at, opted_in_by = excluded.opted_in_by, updated_at = excluded.updated_at",
101106 )
102- .bind(account.id, provider, ciphertext, hint, at, at, at)
107+ .bind(account.id, provider, ciphertext, hint, at, org ? (by?.id ?? null) : null, at, at)
103108 .run();
104- await enqueueMail(this.env, [{ kind: "model_key_added", data: { provider: PROVIDERS[provider].name, hint }, to: { accountId: account.id }, reason: "security_alert", at }]);
105- return { provider, hint, optedInAt: at, createdAt: at, updatedAt: at };
109+ const data = { provider: PROVIDERS[provider].name, hint, ...(org ? { org: account.login, by: by?.login } : {}) };
110+ const to = org ? (await this.env.DB.prepare("SELECT user_id FROM org_members WHERE org_id = ? AND role = 'owner'").bind(account.id).all<{ user_id: string }>()).results.map((r) => r.user_id) : [account.id];
111+ if (to.length) await enqueueMail(this.env, to.map((accountId) => ({ kind: "model_key_added" as const, data, to: { accountId }, reason: "security_alert" as const, at })));
112+ return { provider, hint, optedInAt: at, optedInBy: org ? (by?.login ?? null) : null, createdAt: at, updatedAt: at };
106113 }
107114
108115 async delete(accountId: string, provider: Provider): Promise<boolean> {
@@ -110,7 +117,7 @@
110117 return (r.meta.changes ?? 0) > 0;
111118 }
112119
113- /** The key itself: only for the model proxy, while a task of this person runs. */
120+ /** The key itself: only for the model proxy, inside the request it forwards, while a task this account pays for runs. */
114121 async reveal(accountId: string, provider: Provider): Promise<string | null> {
115122 const row = await this.env.DB.prepare("SELECT ciphertext FROM model_keys WHERE account_id = ? AND provider = ?").bind(accountId, provider).first<{ ciphertext: string }>();
116123 return row ? decryptSecret(this.env.MASTER_KEY, scope(accountId, provider), row.ciphertext) : null;
modifiedapps/diffinery/src/agentic/ledger.ts+52−2View file
@@ -3,7 +3,9 @@
33 /**
44 * A person's model usage across repositories (phase 5a), from the `agent_usage` ledger each repository's
55 * ProjectDO writes when a run ends: for Settings → Model access and `GET /diffinery/model-usage`. Only the
6- * payer's own runs: their tasks, their coordinator's, and reviews in repositories where they turned them on.
6+ * payer's own runs: their tasks, their coordinator's, and reviews in repositories where they turned them on, paid
7+ * with their own keys or subscription. Runs an organization's key paid for (phase 5b) are in its own report
8+ * (orgUsageReport), for its owners.
79 * Repositories are named as they were when each run ended, never by a later name the payer may not be able to read.
810 */
911
@@ -45,7 +47,7 @@
4547 sum(u.output_tokens) AS output, sum(u.cache_read_tokens) AS cache_read, sum(u.cache_write_tokens) AS cache_write,
4648 sum(u.cost_usd) AS cost, sum(u.source = 'runtime') AS reported
4749 FROM agent_usage u
48- WHERE u.person_id = ? AND u.ended_at >= ?
50+ WHERE u.person_id = ?1 AND u.ended_at >= ?2 AND (u.key_owner IS NULL OR u.key_owner = ?1)
4951 GROUP BY u.repo_id, u.repository, u.kind
5052 ORDER BY sum(u.input_tokens + u.output_tokens + u.cache_read_tokens + u.cache_write_tokens) DESC
5153 LIMIT 200`,
@@ -87,3 +89,51 @@
8789 cost_usd: l.costUsd === null ? null : Math.round(l.costUsd * 10_000) / 10_000,
8890 })),
8991 });
92+
93+export interface OrgUsageLine extends UsageLine {
94+ personId: string;
95+ /** The person's login now (owners see their members); null for a deleted account. */
96+ person: string | null;
97+}
98+
99+/** Phase 5b: the runs an organization's keys paid for, per repository, person and kind, for its owners. */
100+export async function orgUsageReport(env: Env, orgId: string, days = USAGE_DAYS_DEFAULT): Promise<{ days: number; since: string; lines: OrgUsageLine[] }> {
101+ const since = new Date(Date.now() - days * 86400_000).toISOString();
102+ const { results } = await env.DB.prepare(
103+ `SELECT u.repo_id, u.repository, u.person_id, CASE WHEN a.deleted_at IS NULL THEN a.login END AS person, u.kind, count(*) AS runs, sum(u.requests) AS requests, sum(u.input_tokens) AS input,
104+ sum(u.output_tokens) AS output, sum(u.cache_read_tokens) AS cache_read, sum(u.cache_write_tokens) AS cache_write,
105+ sum(u.cost_usd) AS cost, sum(u.source = 'runtime') AS reported
106+ FROM agent_usage u LEFT JOIN accounts a ON a.id = u.person_id
107+ WHERE u.key_owner = ? AND u.ended_at >= ?
108+ GROUP BY u.repo_id, u.repository, u.person_id, u.kind
109+ ORDER BY sum(u.input_tokens + u.output_tokens + u.cache_read_tokens + u.cache_write_tokens) DESC
110+ LIMIT 500`,
111+ )
112+ .bind(orgId, since)
113+ .all<{ repo_id: string; repository: string; person_id: string; person: string | null; kind: UsageLine["kind"]; runs: number; requests: number; input: number; output: number; cache_read: number; cache_write: number; cost: number | null; reported: number }>();
114+ return {
115+ days,
116+ since,
117+ lines: results.map((x) => ({
118+ repoId: x.repo_id,
119+ repository: x.repository || null,
120+ personId: x.person_id,
121+ person: x.person,
122+ kind: x.kind,
123+ runs: x.runs,
124+ requests: x.requests,
125+ input: x.input,
126+ output: x.output,
127+ cacheRead: x.cache_read,
128+ cacheWrite: x.cache_write,
129+ costUsd: x.cost,
130+ reportedRuns: x.reported,
131+ })),
132+ };
133+}
134+
135+export const orgUsageReportJson = (r: { days: number; since: string; lines: OrgUsageLine[] }) => ({
136+ days: r.days,
137+ since: r.since,
138+ repositories: r.lines.map((l) => ({ ...usageReportJson({ days: r.days, since: r.since, lines: [l] }).repositories[0]!, person: l.person })),
139+});
addedapps/diffinery/src/agentic/org-policy.ts+346−0View file
@@ -1,0 +1,346 @@
1+import type { Env } from "../env.js";
2+import type { Account, Directory, Repo } from "../lib/directory.js";
3+import { HttpError, now } from "../lib/util.js";
4+import { ModelKeys, PROVIDERS, type Provider } from "./keys.js";
5+import { MAX_RUNNING_MAX, MAX_RUNNING_MIN } from "./project.js";
6+import { RUNTIMES, runtimeLabel, runtimeNeeds, type RuntimeId } from "./runtimes.js";
7+
8+/**
9+ * Organizations' agent policies and model keys (phase 5b): what owners decide for agents in the organization's
10+ * repositories, and whose key pays. Nothing changes for an organization that set nothing, and repositories owned
11+ * by a person have no policy.
12+ *
13+ * Whose key pays is decided when a task starts (a review or coordinator: when its run starts) and kept on the task
14+ * as `key_owner`: the person's id, or the organization's. The model proxy re-checks it on every request, uncached,
15+ * before it decrypts the key (proxyPayer): only a member who is a person (never an agent account) spends the
16+ * organization's key, and only in a repository the organization still owns, under its monthly allowance.
17+ */
18+
19+export type KeysPolicy = "personal" | "either" | "organization";
20+export const KEYS_POLICIES: readonly KeysPolicy[] = ["personal", "either", "organization"];
21+const RUNTIME_IDS = Object.keys(RUNTIMES) as RuntimeId[];
22+const PROVIDER_IDS = Object.keys(PROVIDERS) as Provider[];
23+
24+/** The largest monthly allowance per person on the organization's keys: 10 billion weighted tokens. */
25+export const MONTHLY_TOKENS_MAX = 10_000_000_000;
26+export const MONTHLY_TOKENS_MIN = 10_000;
27+
28+export interface AgentPolicy {
29+ keys: KeysPolicy;
30+ runtimes: RuntimeId[];
31+ providers: Provider[];
32+ /** Most tasks at once per repository; each project's own setting is clamped to it. */
33+ maxRunning: number;
34+ terminals: boolean;
35+ /** Whether terminals may run agent tasks (only while terminals are allowed). */
36+ terminalTasks: boolean;
37+ review: boolean;
38+ /** Most weighted tokens each person may spend on the organization's keys per calendar month (UTC); null for no limit. */
39+ memberMonthlyTokens: number | null;
40+}
41+
42+export const DEFAULT_POLICY: AgentPolicy = {
43+ keys: "personal",
44+ runtimes: [...RUNTIME_IDS],
45+ providers: [...PROVIDER_IDS],
46+ maxRunning: MAX_RUNNING_MAX,
47+ terminals: true,
48+ terminalTasks: true,
49+ review: true,
50+ memberMonthlyTokens: null,
51+};
52+
53+export interface StoredPolicy extends AgentPolicy {
54+ updatedBy: string | null;
55+ updatedAt: string | null;
56+}
57+
58+export async function orgPolicy(env: Env, orgId: string): Promise<StoredPolicy> {
59+ const row = await env.DB.prepare("SELECT p.json, a.login AS by_login, p.updated_at FROM org_agent_policies p LEFT JOIN accounts a ON a.id = p.updated_by WHERE p.org_id = ?")
60+ .bind(orgId)
61+ .first<{ json: string; by_login: string | null; updated_at: string }>();
62+ if (!row) return { ...DEFAULT_POLICY, updatedBy: null, updatedAt: null };
63+ return { ...DEFAULT_POLICY, ...(JSON.parse(row.json) as Partial<AgentPolicy>), updatedBy: row.by_login, updatedAt: row.updated_at };
64+}
65+
66+/** The policy of the organization that owns `repo`, or null for a person's repository. */
67+export async function repoPolicy(env: Env, repo: Pick<Repo, "owner_id" | "owner_kind">): Promise<{ orgId: string; policy: AgentPolicy } | null> {
68+ if (repo.owner_kind !== "org") return null;
69+ return { orgId: repo.owner_id, policy: await orgPolicy(env, repo.owner_id) };
70+}
71+
72+export interface PolicyInput {
73+ keys?: unknown;
74+ runtimes?: unknown;
75+ providers?: unknown;
76+ max_running?: unknown;
77+ terminals?: unknown;
78+ terminal_tasks?: unknown;
79+ review?: unknown;
80+ member_monthly_tokens?: unknown;
81+}
82+
83+const list = <T extends string>(v: unknown, all: readonly T[], name: string): T[] => {
84+ if (!Array.isArray(v) || v.some((x) => typeof x !== "string" || !all.includes(x as T))) throw new HttpError(422, `\`${name}\` must be a list of: ${all.join(", ")}`);
85+ return all.filter((x) => v.includes(x));
86+};
87+
88+const bool = (v: unknown, name: string): boolean => {
89+ if (typeof v !== "boolean") throw new HttpError(422, `\`${name}\` must be true or false`);
90+ return v;
91+};
92+
93+/** A policy from the REST body or the settings form; fields left out keep `current`. */
94+export function parsePolicy(input: PolicyInput, current: AgentPolicy): AgentPolicy {
95+ const p: AgentPolicy = { ...current };
96+ if (input.keys !== undefined) {
97+ if (typeof input.keys !== "string" || !KEYS_POLICIES.includes(input.keys as KeysPolicy)) throw new HttpError(422, `\`keys\` must be ${KEYS_POLICIES.join(", ")}`);
98+ p.keys = input.keys as KeysPolicy;
99+ }
100+ if (input.runtimes !== undefined) p.runtimes = list(input.runtimes, RUNTIME_IDS, "runtimes");
101+ if (input.providers !== undefined) p.providers = list(input.providers, PROVIDER_IDS, "providers");
102+ if (input.max_running !== undefined) {
103+ const n = Number(input.max_running);
104+ if (input.max_running === "" || input.max_running === null || !Number.isInteger(n) || n < MAX_RUNNING_MIN || n > MAX_RUNNING_MAX) throw new HttpError(422, `\`max_running\` must be a whole number from ${MAX_RUNNING_MIN} to ${MAX_RUNNING_MAX}`);
105+ p.maxRunning = n;
106+ }
107+ if (input.terminals !== undefined) p.terminals = bool(input.terminals, "terminals");
108+ if (input.terminal_tasks !== undefined) p.terminalTasks = bool(input.terminal_tasks, "terminal_tasks");
109+ if (input.review !== undefined) p.review = bool(input.review, "review");
110+ if (input.member_monthly_tokens !== undefined) {
111+ const v = input.member_monthly_tokens;
112+ if (v === null || v === "") p.memberMonthlyTokens = null;
113+ else {
114+ const n = Number(v);
115+ if (!Number.isInteger(n) || n < MONTHLY_TOKENS_MIN || n > MONTHLY_TOKENS_MAX) throw new HttpError(422, `\`member_monthly_tokens\` must be null (no limit) or a whole number from ${MONTHLY_TOKENS_MIN} to ${MONTHLY_TOKENS_MAX}`);
116+ p.memberMonthlyTokens = n;
117+ }
118+ }
119+ if (!p.runtimes.length) throw new HttpError(422, "allow at least one runtime");
120+ if (!p.providers.length) throw new HttpError(422, "allow at least one provider");
121+ return p;
122+}
123+
124+export const policyJson = (p: StoredPolicy) => ({
125+ keys: p.keys,
126+ runtimes: p.runtimes,
127+ providers: p.providers,
128+ max_running: p.maxRunning,
129+ terminals: p.terminals,
130+ terminal_tasks: p.terminalTasks,
131+ review: p.review,
132+ member_monthly_tokens: p.memberMonthlyTokens,
133+ updated_by: p.updatedBy,
134+ updated_at: p.updatedAt,
135+});
136+
137+/**
138+ * Save an organization's policy (the caller checked it's an owner, or a site admin), and clamp its repositories'
139+ * "how many at once" now: their ProjectDOs keep the organization's limit, which every task start refreshes too.
140+ */
141+export async function savePolicy(env: Env, dir: Directory, org: Account, me: Account, p: AgentPolicy, wait: (p: Promise<unknown>) => void): Promise<StoredPolicy> {
142+ const at = now();
143+ const json: AgentPolicy = { keys: p.keys, runtimes: p.runtimes, providers: p.providers, maxRunning: p.maxRunning, terminals: p.terminals, terminalTasks: p.terminalTasks, review: p.review, memberMonthlyTokens: p.memberMonthlyTokens };
144+ await env.DB.prepare("INSERT INTO org_agent_policies (org_id, json, updated_by, updated_at) VALUES (?, ?, ?, ?) ON CONFLICT (org_id) DO UPDATE SET json = excluded.json, updated_by = excluded.updated_by, updated_at = excluded.updated_at")
145+ .bind(org.id, JSON.stringify(json), me.id, at)
146+ .run();
147+ // A site administrator acting for the owners is recorded as such, as on the key audits.
148+ const asSiteAdmin = !!me.site_admin && (await dir.orgRole(org.id, me.id)) !== "owner";
149+ wait(dir.audit({ actorId: me.id, actorLogin: me.login, action: "org.agent_policy.update", target: org.login, detail: { ...policyJson({ ...json, updatedBy: null, updatedAt: null }), updated_by: undefined, updated_at: undefined, ...(asSiteAdmin ? { site_admin: true } : {}) } }));
150+ wait(
151+ (async () => {
152+ // Every repository the organization owns, a page at a time.
153+ for (let after = ""; ; ) {
154+ const page = (await env.DB.prepare("SELECT id FROM repos WHERE owner_id = ? AND id > ? ORDER BY id LIMIT 500").bind(org.id, after).all<{ id: string }>()).results;
155+ for (const r of page) await setRepoLimit(env, r.id, p.maxRunning);
156+ if (page.length < 500) break;
157+ after = page[page.length - 1]!.id;
158+ }
159+ })(),
160+ );
161+ return { ...json, updatedBy: me.login, updatedAt: at };
162+}
163+
164+const setRepoLimit = (env: Env, repoId: string, n: number | null) =>
165+ env.PROJECT.get(env.PROJECT.idFromName(repoId))
166+ .setOrgMaxRunning(n)
167+ .catch((e: unknown) => console.error("clamping a project's tasks at once failed", e));
168+
169+/** After a transfer: the repository takes its new owner's limit on tasks at once, or none when a person owns it. */
170+export async function refreshRepoLimit(env: Env, repo: Repo): Promise<void> {
171+ await setRepoLimit(env, repo.id, (await repoPolicy(env, repo))?.policy.maxRunning ?? null);
172+}
173+
174+// ---------------------------------------------------------------- checks when work starts
175+
176+const orgName = async (dir: Directory, orgId: string) => (await dir.account(orgId))?.login ?? "the organization";
177+
178+/** Why the policy refuses a runtime, or the providers a run would use; null when it allows them. */
179+export function policyRefusal(org: string, p: AgentPolicy, runtime: RuntimeId, providers: Provider[]): string | null {
180+ if (!p.runtimes.includes(runtime)) return `${org}'s agent policy doesn't allow ${RUNTIMES[runtime].name}`;
181+ const off = providers.filter((x) => !p.providers.includes(x));
182+ if (off.length) return `${org}'s agent policy doesn't allow ${off.map((x) => PROVIDERS[x].name).join(" or ")}`;
183+ return null;
184+}
185+
186+/** Whether `person` may spend `orgId`'s key: an active person (never an agent account) who is a member, admin or owner. */
187+export async function spendsOrgKey(dir: Directory, orgId: string, person: Account | null): Promise<boolean> {
188+ if (!person || person.kind !== "user" || person.status !== "active") return false;
189+ return !!(await dir.orgRole(orgId, person.id));
190+}
191+
192+export interface Payer {
193+ /** Whose key pays: the person's id, or the organization's. */
194+ keyOwner: string;
195+ providers: Provider[];
196+ /** The organization's limit on tasks at once, for its repositories. */
197+ orgMaxRunning: number | null;
198+}
199+
200+/**
201+ * Whose key a run of `person`'s pays, and the providers it gets, under the repository's policy. `fixed` is the key
202+ * owner a task already has (messages, retries and plan approval keep it); left out, it is decided now. Throws an
203+ * HttpError saying what to do when nobody's key may pay.
204+ */
205+export async function payerFor(env: Env, dir: Directory, repo: Repo, person: Account, runtime: RuntimeId, via: Provider | null, fixed?: string | null): Promise<Payer> {
206+ const keys = new ModelKeys(env);
207+ const needs = runtimeNeeds(runtime, via);
208+ const rp = await repoPolicy(env, repo);
209+ const personal = async (): Promise<Payer> => {
210+ const have = await keys.providers(person.id);
211+ for (const p of needs) if (!have.has(p)) throw new HttpError(422, `${runtimeLabel(runtime, { via })} needs your ${PROVIDERS[p].name} API key: add it in Settings → Model access`);
212+ let providers = runtime === "custom" ? [...have] : needs;
213+ if (rp && runtime === "custom") providers = providers.filter((p) => rp.policy.providers.includes(p));
214+ if (rp && runtime === "custom" && !providers.length) throw new HttpError(403, `${repo.owner_login}'s agent policy allows none of the providers you have a key for`);
215+ return { keyOwner: person.id, providers, orgMaxRunning: rp?.policy.maxRunning ?? null };
216+ };
217+ if (!rp) {
218+ if (fixed && fixed !== person.id) throw new HttpError(403, "this task was paid for by an organization that no longer owns the repository: start a new task");
219+ return personal();
220+ }
221+ const { orgId, policy } = rp;
222+ const refusal = policyRefusal(repo.owner_login, policy, runtime, needs);
223+ if (refusal) throw new HttpError(403, refusal);
224+ const org = async (): Promise<Payer | null> => {
225+ if (!(await spendsOrgKey(dir, orgId, person))) return null;
226+ const have = [...(await keys.providers(orgId))].filter((p) => policy.providers.includes(p));
227+ const providers = runtime === "custom" ? have : needs.filter((p) => have.includes(p));
228+ if (!providers.length || (runtime !== "custom" && providers.length < needs.length)) return null;
229+ return { keyOwner: orgId, providers, orgMaxRunning: policy.maxRunning };
230+ };
231+ if (fixed) {
232+ // A task keeps whose key pays from its start: re-check that it still may.
233+ if (fixed === person.id) {
234+ if (policy.keys === "organization") throw new HttpError(403, `${repo.owner_login} now pays for agents in its repositories only with its own keys: start a new task`);
235+ return personal();
236+ }
237+ if (fixed !== orgId) throw new HttpError(403, `this task was paid for by an organization that no longer owns ${repo.owner_login}/${repo.name}: start a new task`);
238+ if (policy.keys === "personal") throw new HttpError(403, `${repo.owner_login} no longer pays for agents with its keys: start a new task with yours`);
239+ const o = await org();
240+ if (!o) throw new HttpError(403, (await spendsOrgKey(dir, orgId, person)) ? `${repo.owner_login} no longer has the ${needs.map((p) => PROVIDERS[p].name).join(" and ")} key this task uses` : `only members of ${repo.owner_login} can run agents on its keys`);
241+ return o;
242+ }
243+ if (policy.keys === "personal") return personal();
244+ const o = await org();
245+ if (o) return o;
246+ if (policy.keys === "organization") {
247+ if (!(await spendsOrgKey(dir, orgId, person))) throw new HttpError(403, `${repo.owner_login} pays for agents in its repositories with its own keys, which only its members can use`);
248+ throw new HttpError(422, `${repo.owner_login} pays for agents with its own keys, and has no ${needs.map((p) => PROVIDERS[p].name).join(" and ")} key: an owner can add it on the organization's Settings tab`);
249+ }
250+ return personal();
251+}
252+
253+/** The policy's word on web terminals in `repo`: null when allowed. */
254+export async function terminalRefusal(env: Env, repo: Repo, opts: { tasks?: boolean; runtime?: RuntimeId; person?: { dir: Directory; account: Account } }): Promise<string | null> {
255+ const rp = await repoPolicy(env, repo);
256+ if (!rp) return null;
257+ const org = repo.owner_login;
258+ if (!rp.policy.terminals) return `${org}'s agent policy doesn't allow web terminals`;
259+ if (opts.tasks && !rp.policy.terminalTasks) return `${org}'s agent policy doesn't allow agent tasks in terminals`;
260+ // A terminal's task runs on the person's own subscription, with its runtime's provider: the policy's runtimes and providers still hold.
261+ if (opts.runtime) {
262+ const refused = policyRefusal(org, rp.policy, opts.runtime, RUNTIMES[opts.runtime].needs);
263+ if (refused) return refused;
264+ }
265+ // Under "only the organization's keys", only its members run agents in its repositories, terminals included.
266+ if (opts.tasks && opts.person && rp.policy.keys === "organization" && !(await spendsOrgKey(opts.person.dir, repo.owner_id, opts.person.account)))
267+ return `${org}'s agent policy lets only its members run agent tasks in its repositories`;
268+ return null;
269+}
270+
271+/** The policy's word on agent review in `repo`: null when allowed. */
272+export async function reviewRefusal(env: Env, repo: Repo): Promise<string | null> {
273+ const rp = await repoPolicy(env, repo);
274+ return rp && !rp.policy.review ? `${repo.owner_login}'s agent policy doesn't allow agent review` : null;
275+}
276+
277+// ---------------------------------------------------------------- the model proxy's check, every request
278+
279+/**
280+ * Whether a run's request may spend `keyOwner`'s key now. Nothing is cached: a member removed, a repository
281+ * transferred, a policy or key changed applies at the next request. For an organization's key it returns the
282+ * person's monthly allowance, which the proxy books against.
283+ */
284+export async function proxyPayer(
285+ env: Env,
286+ dir: Directory,
287+ run: { repo: Repo; personId: string; keyOwner: string; provider: Provider },
288+): Promise<{ org: { id: string; login: string; cap: number | null } | null } | { refuse: string }> {
289+ const { repo, personId, keyOwner, provider } = run;
290+ const rp = await repoPolicy(env, repo);
291+ if (keyOwner === personId) {
292+ if (!rp) return { org: null };
293+ if (rp.policy.keys === "organization") return { refuse: `${repo.owner_login} now pays for agents in its repositories only with its own keys` };
294+ if (!rp.policy.providers.includes(provider)) return { refuse: `${repo.owner_login}'s agent policy doesn't allow ${PROVIDERS[provider].name}` };
295+ return { org: null };
296+ }
297+ if (!rp || rp.orgId !== keyOwner) return { refuse: `the organization whose key this task uses no longer owns ${repo.owner_login}/${repo.name}` };
298+ const { policy } = rp;
299+ if (policy.keys === "personal") return { refuse: `${repo.owner_login} no longer pays for agents with its keys` };
300+ if (!policy.providers.includes(provider)) return { refuse: `${repo.owner_login}'s agent policy doesn't allow ${PROVIDERS[provider].name}` };
301+ const person = await dir.account(personId);
302+ if (!(await spendsOrgKey(dir, keyOwner, person))) return { refuse: `${person?.login ?? "the person who started this task"} is no longer a member of ${repo.owner_login}, whose key this task uses` };
303+ return { org: { id: keyOwner, login: repo.owner_login, cap: policy.memberMonthlyTokens } };
304+}
305+
306+/** The calendar month (UTC) an organization's allowance counts in, as `2026-10`. */
307+export const spendMonth = (at = new Date()): string => at.toISOString().slice(0, 7);
308+
309+/**
310+ * Book `tokens` (a request's estimate, weighted) against the person's allowance on the organization's keys this
311+ * month, in one statement: false, booking nothing, when it would go over `cap`. Without a cap it always books, so
312+ * the month's count is right if an owner sets one later.
313+ */
314+export async function bookSpend(env: Env, orgId: string, personId: string, tokens: number, cap: number | null): Promise<boolean> {
315+ const month = spendMonth();
316+ if (cap === null) {
317+ await env.DB.prepare("INSERT INTO org_key_spend (org_id, person_id, month, tokens) VALUES (?, ?, ?, ?) ON CONFLICT (org_id, person_id, month) DO UPDATE SET tokens = tokens + excluded.tokens").bind(orgId, personId, month, tokens).run();
318+ return true;
319+ }
320+ const r = await env.DB.prepare(
321+ "INSERT INTO org_key_spend (org_id, person_id, month, tokens) SELECT ?1, ?2, ?3, ?4 WHERE ?4 <= ?5 AND ?5 > 0 ON CONFLICT (org_id, person_id, month) DO UPDATE SET tokens = tokens + excluded.tokens WHERE org_key_spend.tokens < ?5 AND org_key_spend.tokens + excluded.tokens <= ?5",
322+ )
323+ .bind(orgId, personId, month, tokens, cap)
324+ .run();
325+ return (r.meta.changes ?? 0) > 0;
326+}
327+
328+/** Settle a booking: `delta` is what the answer used minus what was booked (negative for a refund). */
329+export async function settleSpend(env: Env, orgId: string, personId: string, delta: number, month = spendMonth()): Promise<void> {
330+ if (!delta) return;
331+ await env.DB.prepare("UPDATE org_key_spend SET tokens = MAX(tokens + ?, 0) WHERE org_id = ? AND person_id = ? AND month = ?").bind(delta, orgId, personId, month).run();
332+}
333+
334+export interface OrgSpend {
335+ personId: string;
336+ login: string | null;
337+ tokens: number;
338+}
339+
340+/** Each person's tokens on the organization's keys this month, most first. */
341+export async function orgSpend(env: Env, orgId: string, month = spendMonth()): Promise<OrgSpend[]> {
342+ const r = await env.DB.prepare("SELECT s.person_id, a.login, s.tokens FROM org_key_spend s LEFT JOIN accounts a ON a.id = s.person_id WHERE s.org_id = ? AND s.month = ? ORDER BY s.tokens DESC LIMIT 500")
343+ .bind(orgId, month)
344+ .all<{ person_id: string; login: string | null; tokens: number }>();
345+ return r.results.map((x) => ({ personId: x.person_id, login: x.login, tokens: x.tokens }));
346+}
addedapps/diffinery/src/agentic/org-web.tsx+329−0View file
@@ -1,0 +1,329 @@
1+/** @jsxImportSource hono/jsx */
2+import type { FC } from "hono/jsx";
3+import { requireRecentSignIn, type AppContext } from "../auth/principal.js";
4+import type { Env } from "../env.js";
5+import type { Account } from "../lib/directory.js";
6+import { HttpError } from "../lib/util.js";
7+import { KCheckbox, KRadio } from "../web/kumo.js";
8+import { Form, Time, act, post, prm, type WebApp } from "../web/ui.js";
9+import { isProvider, ModelKeys, PROVIDERS, type ModelKeyInfo, type Provider } from "./keys.js";
10+import { orgUsageReport, type OrgUsageLine } from "./ledger.js";
11+import { DEFAULT_POLICY, MONTHLY_TOKENS_MAX, MONTHLY_TOKENS_MIN, orgPolicy, orgSpend, parsePolicy, savePolicy, spendMonth, type KeysPolicy, type OrgSpend, type StoredPolicy } from "./org-policy.js";
12+import { MAX_RUNNING_MAX, MAX_RUNNING_MIN } from "./project.js";
13+import { RUNTIMES, type RuntimeId } from "./runtimes.js";
14+import { requirePersonSession } from "./tasks.js";
15+import { UsageCells } from "./tasks-web.js";
16+import { formatTokens } from "./usage.js";
17+
18+/**
19+ * Phase 5b on an organization's Settings tab (owners only): its agent policy, its model keys and what they paid
20+ * for. The REST API has the same under /orgs/{org}/diffinery (agentic/api.ts).
21+ */
22+
23+export interface OrgAgentData {
24+ policy: StoredPolicy;
25+ keys: ModelKeyInfo[];
26+ usage: { days: number; lines: OrgUsageLine[] };
27+ spend: OrgSpend[];
28+}
29+
30+export async function orgAgentData(env: Env, orgId: string): Promise<OrgAgentData> {
31+ const [policy, keys, usage, spend] = await Promise.all([orgPolicy(env, orgId), new ModelKeys(env).list(orgId), orgUsageReport(env, orgId), orgSpend(env, orgId)]);
32+ return { policy, keys, usage, spend };
33+}
34+
35+const KEYS_LABEL: Record<KeysPolicy, { label: string; description: string }> = {
36+ personal: { label: "Each person's own keys", description: "The default: people run agents with their own keys, as anywhere else." },
37+ either: { label: "The organization's keys, when it has them", description: "Members' runs use the organization's key for the provider they need, otherwise their own. Outside collaborators always use their own." },
38+ organization: { label: "Only the organization's keys", description: "Personal keys are never used in the organization's repositories. Only members can run agents here; outside collaborators can't." },
39+};
40+
41+export const OrgAgentSettings: FC<{ org: Account; data: OrgAgentData; t: string }> = ({ org, data, t }) => {
42+ const base = `/${org.login}/-/org`;
43+ const p = data.policy;
44+ const keys = new Map(data.keys.map((k) => [k.provider, k]));
45+ const cap = p.memberMonthlyTokens;
46+ return (
47+ <>
48+ <h2 id="agent-policy">Agents</h2>
49+ <Form action={`${base}/agent-policy`} t={t} class="box">
50+ <div class="head">
51+ <h2>Agent policy</h2>
52+ {p.updatedAt ? (
53+ <span class="muted small">
54+ Changed by {p.updatedBy ?? "someone"} <Time at={p.updatedAt} />
55+ </span>
56+ ) : (
57+ <span class="badge">Defaults</span>
58+ )}
59+ </div>
60+ <div class="body">
61+ <p class="muted">
62+ How agents may work in {org.login}'s repositories, and whose key pays. With nothing changed, members use their own keys and everything is allowed. See <a href="/docs/agents/organizations">Agents in organizations</a>.
63+ </p>
64+ <div class="field">
65+ <label>Whose key pays</label>
66+ <KRadio name="keys" label="Whose key pays" value={p.keys} items={(Object.keys(KEYS_LABEL) as KeysPolicy[]).map((k) => ({ value: k, label: KEYS_LABEL[k].label, description: KEYS_LABEL[k].description }))} />
67+ <div class="hint">A task keeps whose key pays from when it starts. Every model request checks again that the person is still a member, the repository still the organization's, and the key still allowed; if not, the task stops.</div>
68+ </div>
69+ <div class="field">
70+ <label>Allowed runtimes</label>
71+ {(Object.keys(RUNTIMES) as RuntimeId[]).map((r) => (
72+ <div>
73+ <KCheckbox name="runtimes" value={r} checked={p.runtimes.includes(r)} label={RUNTIMES[r].name} />
74+ </div>
75+ ))}
76+ </div>
77+ <div class="field">
78+ <label>Allowed providers</label>
79+ {(Object.keys(PROVIDERS) as Provider[]).map((x) => (
80+ <div>
81+ <KCheckbox name="providers" value={x} checked={p.providers.includes(x)} label={PROVIDERS[x].name} />
82+ </div>
83+ ))}
84+ <div class="hint">Checked when tasks, reviews, coordinators and terminals start, and on every model request.</div>
85+ </div>
86+ <div class="field">
87+ <label for="org-max-running">Most tasks at once per repository</label>
88+ <input id="org-max-running" name="max_running" type="number" min={MAX_RUNNING_MIN} max={MAX_RUNNING_MAX} step={1} required style="max-width:8em" value={String(p.maxRunning)} />
89+ <div class="hint">
90+ From {MAX_RUNNING_MIN} to {MAX_RUNNING_MAX}. Each project's own "tasks at once" is held to it; more wait in the queue.
91+ </div>
92+ </div>
93+ <div class="field">
94+ <label>Web terminals</label>
95+ <div>
96+ <KCheckbox name="terminals" checked={p.terminals} label="Allow web terminals" />
97+ </div>
98+ <div>
99+ <KCheckbox name="terminal_tasks" checked={p.terminalTasks} label="Let terminals run agent tasks" />
100+ </div>
101+ <div class="hint">Turning terminals off closes none that are open, but no new ones open and no new tasks start in them.</div>
102+ </div>
103+ <div class="field">
104+ <label>Agent review</label>
105+ <KCheckbox name="review" checked={p.review} label="Allow agent review" />
106+ <div class="hint">Turning it off stops new reviews; repository admins' settings stay for when it's on again.</div>
107+ </div>
108+ <div class="field">
109+ <label for="org-monthly">Monthly tokens per person on the organization's keys</label>
110+ <input id="org-monthly" name="member_monthly_tokens" type="number" min={MONTHLY_TOKENS_MIN} max={MONTHLY_TOKENS_MAX} step={1} style="max-width:14em" value={cap === null ? "" : String(cap)} placeholder="No limit" />
111+ <div class="hint">
112+ Leave empty for no limit. Each request books what it may use before it goes to the provider; past the limit the person's runs on the organization's keys stop until next month (UTC). Cache reads count a tenth, as in budgets.
113+ </div>
114+ </div>
115+ <button class="btn primary">Save agent policy</button>
116+ </div>
117+ </Form>
118+
119+ <div class="box" id="model-keys">
120+ <div class="head">
121+ <h2>Model keys</h2>
122+ <span class="badge">{data.keys.length ? `${data.keys.length} saved` : "None"}</span>
123+ </div>
124+ <div class="body">
125+ <p class="muted">
126+ Keys the organization pays with, when its policy says so. Only owners see or change them, and every owner gets an email when one is added. Diffinery keeps them encrypted, never shows them again, and never puts them in a sandbox, container or terminal: the model proxy adds the key to each request itself. Use dedicated keys with a spending limit.
127+ </p>
128+ {(Object.keys(PROVIDERS) as Provider[]).map((id) => {
129+ const info = PROVIDERS[id];
130+ const k = keys.get(id);
131+ return (
132+ <div class="mb">
133+ <div class="row">
134+ <b class="grow">{info.name}</b>
135+ {k ? (
136+ <>
137+ <span class="mono">…{k.hint}</span>
138+ <span class="muted small">
139+ terms accepted by {k.optedInBy ?? "an owner"} <Time at={k.optedInAt} />
140+ </span>
141+ <Form action={`${base}/model-keys/${id}/delete`} t={t} class="inline" confirm={`Delete ${org.login}'s ${info.name} key? Running tasks that use it stop getting model access.`}>
142+ <button class="btn sm danger">Delete</button>
143+ </Form>
144+ </>
145+ ) : (
146+ <span class="muted small">No key</span>
147+ )}
148+ </div>
149+ <details>
150+ <summary class="small">{k ? "Replace" : "Add a key"}</summary>
151+ <Form action={`${base}/model-keys/${id}`} t={t}>
152+ <div class="field">
153+ <label for={`org-key-${id}`}>{org.login}'s {info.name} API key</label>
154+ <input id={`org-key-${id}`} name="key" type="password" autocomplete="off" required placeholder={id === "anthropic" ? "sk-ant-…" : id === "openrouter" ? "sk-or-…" : "sk-…"} />
155+ <div class="hint">
156+ Create one at <a href={info.keysUrl} rel="noopener noreferrer" target="_blank">{info.keysUrl.replace("https://", "")}</a>.
157+ </div>
158+ </div>
159+ <div class="field">
160+ <KCheckbox
161+ name="accept"
162+ label={
163+ <>
164+ {org.login} uses {info.name} through Diffinery under its own {info.name} account and its terms ({info.terms.map((x, i) => (
165+ <>
166+ {i ? ", " : ""}
167+ <a href={x.url} rel="noopener noreferrer" target="_blank">
168+ {x.label}
169+ </a>
170+ </>
171+ ))}
172+ ), which I accept for it. Usage is billed to its account. Diffinery only provides the ability.
173+ </>
174+ }
175+ />
176+ </div>
177+ <button class="btn primary">{k ? "Replace key" : "Save key"}</button>
178+ </Form>
179+ </details>
180+ </div>
181+ );
182+ })}
183+ </div>
184+ </div>
185+
186+ <div class="box" id="org-usage">
187+ <div class="head">
188+ <h2>Paid with the organization's keys</h2>
189+ </div>
190+ <div class="body">
191+ <h3>This month ({spendMonth()}, UTC)</h3>
192+ {data.spend.length ? (
193+ <div class="xscroll">
194+ <table class="tbl">
195+ <thead>
196+ <tr>
197+ <th>Person</th>
198+ <th class="r">Tokens</th>
199+ <th class="r">Allowance</th>
200+ </tr>
201+ </thead>
202+ <tbody>
203+ {data.spend.map((x) => (
204+ <tr>
205+ <td>{x.login ? <a href={`/${x.login}`}>{x.login}</a> : <span class="muted">Deleted account</span>}</td>
206+ <td class="r">{formatTokens(x.tokens)}</td>
207+ <td class="r">{cap === null ? "No limit" : `${Math.min(100, Math.round((x.tokens / cap) * 100))}% of ${formatTokens(cap)}`}</td>
208+ </tr>
209+ ))}
210+ </tbody>
211+ </table>
212+ </div>
213+ ) : (
214+ <p class="muted">Nobody has used the organization's keys this month.</p>
215+ )}
216+ <h3 class="mt">Runs in the last {data.usage.days} days</h3>
217+ {data.usage.lines.length ? (
218+ <div class="xscroll">
219+ <table class="tbl">
220+ <thead>
221+ <tr>
222+ <th>Repository</th>
223+ <th>Person</th>
224+ <th>For</th>
225+ <th class="r">Runs</th>
226+ <th class="r">Requests</th>
227+ <th class="r">Input</th>
228+ <th class="r">Output</th>
229+ <th class="r">Cache read / write</th>
230+ <th class="r">Cost</th>
231+ </tr>
232+ </thead>
233+ <tbody>
234+ {data.usage.lines.map((l) => (
235+ <tr>
236+ <td>{l.repository ? <a href={`/${l.repository}/tasks`}>{l.repository}</a> : <span class="muted">Unknown</span>}</td>
237+ <td>{l.person ?? <span class="muted">Deleted account</span>}</td>
238+ <td>{l.kind === "task" ? "Tasks" : l.kind === "review" ? "Agent reviews" : "Coordinator"}</td>
239+ <td class="r">{l.runs}</td>
240+ <UsageCells u={l} />
241+ </tr>
242+ ))}
243+ </tbody>
244+ </table>
245+ </div>
246+ ) : (
247+ <p class="muted">No runs used the organization's keys in the last {data.usage.days} days.</p>
248+ )}
249+ <p class="muted small mt">
250+ Diffinery counts each request from the provider's answer. Tokens this month are weighted as budgets weigh them (a cache read counts a tenth). Also at <code>GET /api/v3/orgs/{org.login}/diffinery/model-usage</code>.
251+ </p>
252+ </div>
253+ </div>
254+ </>
255+ );
256+};
257+
258+// ---------------------------------------------------------------- posts
259+
260+/** An owner of the organization (or a site admin, audited as themselves) in a browser session. */
261+async function orgOwner(c: AppContext) {
262+ const me = requirePersonSession(c);
263+ const dir = c.get("dir");
264+ const org = await dir.accountByLogin(prm(c, "owner"));
265+ if (!org || org.kind !== "org" || org.status === "disabled") throw new HttpError(404, "not found");
266+ const role = await dir.orgRole(org.id, me.id);
267+ if (role !== "owner" && !me.site_admin) throw new HttpError(403, "organization owners only");
268+ return { me, org, dir, asSiteAdmin: role !== "owner", back: `/${org.login}?tab=settings` };
269+}
270+
271+export function registerOrgAgents(web: WebApp): void {
272+ post(web, "/:owner/-/org/agent-policy", async (c, f) => {
273+ const { me, org, dir, back } = await orgOwner(c);
274+ return act(
275+ c,
276+ `${back}#agent-policy`,
277+ async () => {
278+ const monthly = f.s("member_monthly_tokens").trim();
279+ const p = parsePolicy(
280+ {
281+ keys: f.s("keys") || DEFAULT_POLICY.keys,
282+ runtimes: f.all("runtimes"),
283+ providers: f.all("providers"),
284+ max_running: f.s("max_running"),
285+ terminals: f.all("terminals").length > 0,
286+ terminal_tasks: f.all("terminal_tasks").length > 0,
287+ review: f.all("review").length > 0,
288+ member_monthly_tokens: monthly === "" ? null : monthly,
289+ },
290+ await orgPolicy(c.env, org.id),
291+ );
292+ await savePolicy(c.env, dir, org, me, p, (x) => c.executionCtx.waitUntil(x));
293+ },
294+ "Agent policy saved.",
295+ );
296+ });
297+
298+ post(web, "/:owner/-/org/model-keys/:provider", async (c, f) => {
299+ const { me, org, dir, asSiteAdmin, back } = await orgOwner(c);
300+ const provider = prm(c, "provider");
301+ if (!isProvider(provider)) throw new HttpError(404, "not found");
302+ return act(
303+ c,
304+ `${back}#model-keys`,
305+ async () => {
306+ requireRecentSignIn(c, back);
307+ const k = await new ModelKeys(c.env).set(org, provider, f.s("key"), f.all("accept").length > 0, me);
308+ c.executionCtx.waitUntil(dir.audit({ actorId: me.id, actorLogin: me.login, action: "org.model_key.set", target: org.login, detail: { provider, hint: k.hint, opted_in_at: k.optedInAt, ...(asSiteAdmin ? { site_admin: true } : {}) } }));
309+ },
310+ `${org.login}'s ${PROVIDERS[provider].name} key saved.`,
311+ );
312+ });
313+
314+ post(web, "/:owner/-/org/model-keys/:provider/delete", async (c) => {
315+ const { me, org, dir, asSiteAdmin, back } = await orgOwner(c);
316+ const provider = prm(c, "provider");
317+ if (!isProvider(provider)) throw new HttpError(404, "not found");
318+ return act(
319+ c,
320+ `${back}#model-keys`,
321+ async () => {
322+ requireRecentSignIn(c, back);
323+ await new ModelKeys(c.env).delete(org.id, provider);
324+ c.executionCtx.waitUntil(dir.audit({ actorId: me.id, actorLogin: me.login, action: "org.model_key.delete", target: org.login, detail: { provider, ...(asSiteAdmin ? { site_admin: true } : {}) } }));
325+ },
326+ `${org.login}'s ${PROVIDERS[provider].name} key deleted.`,
327+ );
328+ });
329+}
modifiedapps/diffinery/src/agentic/project-web.tsx+3−1View file
@@ -288,7 +288,8 @@
288288 ) : null}
289289 </div>
290290 <p class="muted small">
291- Up to <b>{board.settings.maxRunning}</b> tasks run at once; more wait in the queue.{" "}
291+ Up to <b>{board.settings.orgMaxRunning !== null && board.settings.orgMaxRunning < board.settings.maxRunning ? board.settings.orgMaxRunning : board.settings.maxRunning}</b> tasks run at once
292+ {board.settings.orgMaxRunning !== null && board.settings.orgMaxRunning < board.settings.maxRunning ? ", the organization's limit" : ""}; more wait in the queue.{" "}
292293 {board.settings.instructions ? "Every task follows the project instructions." : "No project instructions."}
293294 </p>
294295 </section>
@@ -504,6 +505,7 @@
504505 <input id="max_running" name="max_running" type="number" min={MAX_RUNNING_MIN} max={MAX_RUNNING_MAX} step={1} required style="max-width:8em" value={values?.max_running ?? String(s.maxRunning)} />
505506 <div class="hint">
506507 How many tasks run at the same time, from {MAX_RUNNING_MIN} to {MAX_RUNNING_MAX}. Tasks started beyond it wait in the queue and start, oldest first, as running ones finish. Agent reviews don't count and never wait.
508+ {s.orgMaxRunning !== null && s.orgMaxRunning < s.maxRunning ? ` The organization's agent policy allows at most ${s.orgMaxRunning}, so ${s.orgMaxRunning} run at once.` : ""}
507509 </div>
508510 </div>
509511 <div class="field">
modifiedapps/diffinery/src/agentic/project.ts+2−0View file
@@ -18,6 +18,8 @@
1818
1919 export interface ProjectSettings {
2020 maxRunning: number;
21+ /** Phase 5b: the owning organization's limit on tasks at once, which clamps maxRunning (null: none). */
22+ orgMaxRunning: number | null;
2123 /** Text every task's prompt carries, set by repository admins. */
2224 instructions: string;
2325 /** Whether new tasks plan first: never by default, by default (people can untick it), or always. */
modifiedapps/diffinery/src/agentic/projects.ts+10−8View file
@@ -8,7 +8,8 @@
88 import { BOARD, boardState, isPlanFirst, MAX_INSTRUCTIONS, MAX_MEMORY, MAX_POST, validMaxRunning, type BoardState, type MemoryRow, type PostRow, type ProjectSettings, type PullState } from "./project.js";
99 import type { TaskRow } from "./ProjectDO.js";
1010 import { isRuntime, MAX_COMMAND } from "./runtimes.js";
11-import { modelRoute, projectStub, providersFor, requirePersonSession, starter, taskJson } from "./tasks.js";
11+import { modelRoute, projectStub, requirePersonSession, starter, taskJson } from "./tasks.js";
12+import { payerFor } from "./org-policy.js";
1213
1314 /**
1415 * The project layer over a repository (agentic/project.ts): the board, the
@@ -82,8 +83,8 @@
8283 /** Split a task's waiting plan into tasks: the person who started it hands it to their coordinator. */
8384 export async function splitPlan(c: AppContext, l: Loaded, number: number): Promise<{ post: number; task: number; status: "started" | "queued" }> {
8485 const { person } = await starter(c, l, number);
85- const { s, providers } = await coordinatorFor(c, l, person);
86- const out = await projectStub(c.env, l.repo.id).splitPlan(number, s, providers, { id: l.repo.id, base: l.repo.default_branch });
86+ const { s, providers, keyOwner } = await coordinatorFor(c, l, person);
87+ const out = await projectStub(c.env, l.repo.id).splitPlan(number, s, providers, { id: l.repo.id, base: l.repo.default_branch }, keyOwner);
8788 c.executionCtx.waitUntil(c.get("dir").audit({ actorId: person.id, actorLogin: person.login, action: "task.plan_split", target: `${l.repo.owner_login}/${l.repo.name}#${number}`, detail: { post: out.post, coordinator_task: out.task } }));
8889 return out;
8990 }
@@ -107,12 +108,13 @@
107108 if (e instanceof TaskAccessError) throw new HttpError(403, e.message);
108109 throw e;
109110 }
110- return { s, providers: await providersFor(c.env, person, s.runtime, s.via) };
111+ const { providers, keyOwner } = await payerFor(c.env, c.get("dir"), l.repo, person, s.runtime, s.via);
112+ return { s, providers, keyOwner };
111113 }
112114
113115 async function handleAs(c: AppContext, l: Loaded, person: Account, n: number) {
114- const { s, providers } = await coordinatorFor(c, l, person);
115- const out = await projectStub(c.env, l.repo.id).handlePost(n, s, providers, { id: l.repo.id, base: l.repo.default_branch });
116+ const { s, providers, keyOwner } = await coordinatorFor(c, l, person);
117+ const out = await projectStub(c.env, l.repo.id).handlePost(n, s, providers, { id: l.repo.id, base: l.repo.default_branch }, keyOwner);
116118 c.executionCtx.waitUntil(c.get("dir").audit({ actorId: person.id, actorLogin: person.login, action: "project.coordinate", target: `${l.repo.owner_login}/${l.repo.name}`, detail: { post: n, task: out.task, agent: s.agent.login, runtime: s.runtime } }));
117119 return out;
118120 }
@@ -160,7 +162,7 @@
160162 throw e;
161163 }
162164 const route = modelRoute(runtime, input.provider, input.model);
163- await providersFor(c.env, me, runtime, route.via);
165+ await payerFor(c.env, c.get("dir"), l.repo, me, runtime, route.via);
164166 const tokensPerRun = tokens(input.tokens_per_run, current?.tokensPerRun ?? COORDINATOR_TOKENS_DEFAULT, "tokens_per_run");
165167 const tokensPerTask = tokens(input.tokens_per_task, current?.tokensPerTask ?? TASK_TOKENS_DEFAULT, "tokens_per_task");
166168 if (input.accept !== true) throw new HttpError(422, "confirm that your coordinator, and the tasks it starts, use your model key");
@@ -235,7 +237,7 @@
235237
236238 export const memoryJson = (m: MemoryRow | null) => (m ? { body: m.body, version: m.n, updated_by: { login: m.by_login, type: m.by_kind === "person" ? "User" : "Bot" }, updated_at: m.at } : { body: "", version: null, updated_by: null, updated_at: null });
237239
238-export const settingsJson = (s: ProjectSettings) => ({ max_running: s.maxRunning, instructions: s.instructions, plan_first: s.planFirst });
240+export const settingsJson = (s: ProjectSettings) => ({ max_running: s.maxRunning, ...(s.orgMaxRunning !== null ? { org_max_running: s.orgMaxRunning } : {}), instructions: s.instructions, plan_first: s.planFirst });
239241
240242 export function boardJson(base: string, repoFull: string, b: Board) {
241243 return {
modifiedapps/diffinery/src/agentic/proxy.ts+143−58View file
@@ -2,7 +2,9 @@
22 import { Directory } from "../lib/directory.js";
33 import { taskPersonAllowed } from "./access.js";
44 import { isProvider, ModelKeys, PROVIDERS, type Provider } from "./keys.js";
5-import { NO_USAGE, prepareRequest, readBody, readUsage } from "./usage.js";
5+import { bookSpend, proxyPayer, settleSpend, spendMonth } from "./org-policy.js";
6+import { freeSlot, MAX_IN_FLIGHT, takeSlot } from "./inflight.js";
7+import { meterUsage, NO_USAGE, prepareRequest, readBody, unreadCharge, weighted } from "./usage.js";
68
79 /** The largest model request the proxy forwards (it reads each one to charge it). */
810 const MAX_REQUEST = 32 * 1024 * 1024;
@@ -10,8 +12,10 @@
1012 /**
1113 * The model proxy: `/-/model/<provider>/v1/...`. A task's runtime calls it
1214 * with the run's token (`dmt_...`, as its API key) and the proxy forwards the
13- * request to the provider with the person's own key, added here. So the key
14- * never enters the sandbox, and the token stops working when the run ends.
15+ * request to the provider with the key of whoever pays (the person's own, or
16+ * under its policy an organization's: org-policy.ts), decrypted here for this
17+ * request only. So no key ever enters a sandbox, container or terminal, and the
18+ * token stops working when the run ends.
1519 *
1620 * It runs before the app's identity middleware: its token isn't an account
1721 * credential and works nowhere else.
@@ -58,6 +62,9 @@
5862 /** OpenRouter's app attribution and request options: only Diffinery sets these, never the sandbox. */
5963 const ATTRIBUTION = /^(x-openrouter-.*|x-title|http-referer|referer)$/i;
6064
65+/** Headers that pick an organization or project within a provider account (OpenAI's). */
66+const ACCOUNT_SCOPE = /^openai-(organization|project)$/i;
67+
6168 const err = (status: number, message: string) =>
6269 new Response(JSON.stringify({ type: "error", error: { type: status === 401 ? "authentication_error" : status === 404 ? "not_found_error" : status === 429 ? "rate_limit_error" : "permission_error", message } }), { status, headers: { "content-type": "application/json" } });
6370
@@ -91,59 +98,137 @@
9198 body = prepared.body;
9299 }
93100 const { estimate } = prepared;
94- const run = await project.modelAuth(token, { provider, estimate, reviewRefuse: prepared.reviewRefuse, model: prepared.model, routing: prepared.routing });
95- if (!run) return err(401, "this model token isn't valid: its task run has ended");
96- if ("limit" in run) return err(429, run.limit);
97- if ("refuse" in run) return err(403, run.refuse);
98- if ("busy" in run) {
99- const res = err(429, run.busy);
100- res.headers.set("retry-after", "60");
101- return res;
102- }
103- // Runs with a budget (reviews, coordinators and the tasks they start) send the capped, tool-stripped body their booking assumes.
104- if (run.bounded && prepared.reviewBody) body = prepared.reviewBody;
105- // A booked request that never reaches the provider, or that it answers with an error, costs nothing.
106- const refund = () => (estimate ? c.executionCtx.waitUntil(project.chargeUsage(run.task, run.run, NO_USAGE, estimate).catch((e) => console.error("refunding a model request failed", e))) : undefined);
107- // The person must still be allowed to run the task, on every request; otherwise the task stops.
108- const dir = new Directory(env);
109- const [repo, agent] = await Promise.all([dir.repoById(run.repoId), dir.account(run.agentId)]);
110- if (!repo || !agent || !(await taskPersonAllowed(dir, run.personId, agent, repo))) {
111- refund();
112- c.executionCtx.waitUntil(project.stop(run.task, "Diffinery", "Stopped: the person who started the task can no longer run it.").then(() => undefined, () => undefined));
113- return err(403, "the person who started this task can no longer run it");
114- }
115- const key = await new ModelKeys(env).reveal(run.personId, provider);
116- if (!key) {
117- refund();
118- return err(403, `no ${PROVIDERS[provider].name} key in Settings → Model access`);
119- }
120- const headers = new Headers();
121- for (const [k, v] of c.req.raw.headers) if (!DROP.has(k.toLowerCase()) && !ATTRIBUTION.test(k)) headers.set(k, v);
122- if (provider === "anthropic") headers.set("x-api-key", key);
123- else headers.set("authorization", `Bearer ${key}`);
124- if (provider === "openrouter") {
125- // OpenRouter's app attribution headers: Diffinery, not whatever the sandbox claims.
126- headers.set("http-referer", env.PUBLIC_URL);
127- headers.set("x-title", "Diffinery");
128- }
129- const base = (provider === "anthropic" ? env.MODEL_UPSTREAM_ANTHROPIC : provider === "openai" ? env.MODEL_UPSTREAM_OPENAI : env.MODEL_UPSTREAM_OPENROUTER) || UPSTREAM[provider];
130- // If the provider can't be reached, the booking stays: it may have started the answer.
131- const res = await fetch(`${base}${rest}${url.search}`, { method: c.req.method, headers, body, redirect: "manual" });
132- const out = new Headers(res.headers);
133- out.delete("set-cookie");
134- if (!res.body || !res.ok) {
135- refund();
136- return new Response(res.body, { status: res.status, headers: out });
137- }
138- if (!estimate) return new Response(res.body, { status: res.status, headers: out });
139- // Settle the booking to the tokens the provider reports, read from a copy of the response as it streams back.
140- // An answer that reports none is charged its estimate; one whose usage can't be read is never settled, and so
141- // stays charged its estimate too.
142- const [agentCopy, meter] = res.body.tee();
143- c.executionCtx.waitUntil(
144- readUsage(meter, res.headers.get("content-type") ?? "")
145- .then((u) => project.chargeUsage(run.task, run.run, u ?? estimate, estimate))
146- .catch((e) => console.error("reading model usage failed", e)),
147- );
148- return new Response(agentCopy, { status: res.status, headers: out });
101+ // The leases a limit on requests at once gave this request (inflight.ts), freed once it is settled.
102+ const slots: string[] = [];
103+ const free = () => {
104+ for (const id of slots.splice(0)) c.executionCtx.waitUntil(freeSlot(env, id).catch((e) => console.error("freeing a model request's slot failed", e)));
105+ };
106+ // Everything from here, the provider's answer and its settlement included, runs under one waitUntil registered
107+ // now: if the sandbox drops its connection while the Worker waits, the request is still settled and its slots freed.
108+ let settling: Promise<unknown> = Promise.resolve();
109+ const work = (async (): Promise<Response> => {
110+ const run = await project.modelAuth(token, { provider, estimate, reviewRefuse: prepared.reviewRefuse, model: prepared.model, routing: prepared.routing });
111+ if (!run) return err(401, "this model token isn't valid: its task run has ended");
112+ if ("limit" in run) return err(429, run.limit);
113+ if ("refuse" in run) return err(403, run.refuse);
114+ if ("busy" in run) {
115+ const res = err(429, run.busy);
116+ res.headers.set("retry-after", "60");
117+ return res;
118+ }
119+ // Runs held to their bookings (reviews, coordinators and the tasks they start, and runs on an organization's key, whose
120+ // monthly allowance depends on them) send the capped, tool-stripped body their booking assumes.
121+ const limited = run.bounded || run.orgKey;
122+ if (limited && prepared.reviewBody) body = prepared.reviewBody;
123+ // A booked request that never reaches the provider, or that it answers with an error, costs nothing.
124+ // `spend` is the booking against the person's monthly allowance on an organization's key, once made.
125+ let spend: { org: string; tokens: number; month: string } | null = null;
126+ const refund = (unsent = true) => {
127+ if (estimate || unsent) c.executionCtx.waitUntil(project.chargeUsage(run.task, run.run, NO_USAGE, estimate, unsent).catch((e) => console.error("refunding a model request failed", e)));
128+ if (spend) c.executionCtx.waitUntil(settleSpend(env, spend.org, run.personId, -spend.tokens, spend.month).catch((e) => console.error("refunding an organization's allowance failed", e)));
129+ free();
130+ };
131+ const busy = (why: string) => {
132+ refund();
133+ const res = err(429, why);
134+ res.headers.set("retry-after", "5");
135+ return res;
136+ };
137+ // A stored response can't be read back on these runs: on an organization's key it could be anyone's in that account.
138+ if (limited && c.req.method === "GET" && rest.startsWith("/v1/responses")) {
139+ refund();
140+ return err(403, `reading stored responses isn't available to agent reviews, runs with a token budget and runs on an organization's key`);
141+ }
142+ // A run held to a budget has at most MAX_IN_FLIGHT generating requests at once.
143+ if (run.bounded && estimate) {
144+ const slot = await takeSlot(env, `run:${run.repoId}:${run.task}:${run.run}`);
145+ if (!slot) return busy(`this run already has ${MAX_IN_FLIGHT} model requests on their way; try again when one is answered`);
146+ slots.push(slot);
147+ }
148+ const stop = (why: string) => c.executionCtx.waitUntil(project.stop(run.task, "Diffinery", `Stopped: ${why}.`).then(() => undefined, () => undefined));
149+ // The person must still be allowed to run the task, on every request; otherwise the task stops.
150+ const dir = new Directory(env);
151+ const [repo, agent] = await Promise.all([dir.repoById(run.repoId), dir.account(run.agentId)]);
152+ if (!repo || !agent || !(await taskPersonAllowed(dir, run.personId, agent, repo))) {
153+ refund();
154+ stop("the person who started the task can no longer run it");
155+ return err(403, "the person who started this task can no longer run it");
156+ }
157+ // Whoever pays must still be allowed to, on every request and uncached (phase 5b): a member removed from the
158+ // organization, a repository transferred out of it, or a policy or key changed applies at the next request.
159+ const payer = await proxyPayer(env, dir, { repo, personId: run.personId, keyOwner: run.keyOwner, provider });
160+ if ("refuse" in payer) {
161+ refund();
162+ stop(payer.refuse);
163+ return err(403, payer.refuse);
164+ }
165+ if (payer.org) {
166+ // Under an allowance, a person has at most MAX_IN_FLIGHT generating requests at once on the organization's keys,
167+ // across all their runs: what an image or PDF bills isn't in the booking, so this bounds how far they can go past it.
168+ if (payer.org.cap !== null && estimate) {
169+ const slot = await takeSlot(env, `org:${payer.org.id}:${run.personId}`);
170+ if (!slot) return busy(`${MAX_IN_FLIGHT} of your model requests on ${payer.org.login}'s keys are already on their way; try again when one is answered`);
171+ slots.push(slot);
172+ }
173+ const tokens = estimate ? weighted(estimate) : 0;
174+ const month = spendMonth();
175+ if (!(await bookSpend(env, payer.org.id, run.personId, tokens, payer.org.cap))) {
176+ const why = `this month's allowance of ${payer.org.cap!.toLocaleString("en-US")} tokens per person on ${payer.org.login}'s keys is used up`;
177+ refund();
178+ stop(why);
179+ return err(403, why);
180+ }
181+ spend = { org: payer.org.id, tokens, month };
182+ }
183+ // Decrypted here, for this request only.
184+ const key = await new ModelKeys(env).reveal(run.keyOwner, provider);
185+ if (!key) {
186+ refund();
187+ return err(403, payer.org ? `${payer.org.login} has no ${PROVIDERS[provider].name} key any more` : `no ${PROVIDERS[provider].name} key in Settings → Model access`);
188+ }
189+ const headers = new Headers();
190+ // On an organization's key, the run doesn't choose the organization or project within the provider's account.
191+ for (const [k, v] of c.req.raw.headers) if (!DROP.has(k.toLowerCase()) && !ATTRIBUTION.test(k) && !(payer.org && ACCOUNT_SCOPE.test(k))) headers.set(k, v);
192+ if (provider === "anthropic") headers.set("x-api-key", key);
193+ else headers.set("authorization", `Bearer ${key}`);
194+ if (provider === "openrouter") {
195+ // OpenRouter's app attribution headers: Diffinery, not whatever the sandbox claims.
196+ headers.set("http-referer", env.PUBLIC_URL);
197+ headers.set("x-title", "Diffinery");
198+ }
199+ const base = (provider === "anthropic" ? env.MODEL_UPSTREAM_ANTHROPIC : provider === "openai" ? env.MODEL_UPSTREAM_OPENAI : env.MODEL_UPSTREAM_OPENROUTER) || UPSTREAM[provider];
200+ // If the provider can't be reached, the booking stays: it may have started the answer.
201+ const res = await fetch(`${base}${rest}${url.search}`, { method: c.req.method, headers, body, redirect: "manual" });
202+ const out = new Headers(res.headers);
203+ out.delete("set-cookie");
204+ if (!res.body || !res.ok) {
205+ refund(false);
206+ return new Response(res.body, { status: res.status, headers: out });
207+ }
208+ if (!estimate) {
209+ free();
210+ return new Response(res.body, { status: res.status, headers: out });
211+ }
212+ // Settle the booking to the tokens the provider reports, read from a copy of the response as it streams back.
213+ // An answer that reports none, or with a part too large to read for usage, is charged its estimate, or on a run
214+ // held to its bookings unreadCharge; one whose reading fails is never settled, and so stays charged its estimate.
215+ const [agentCopy, meter] = res.body.tee();
216+ const booked = spend;
217+ const sent = body === undefined ? 0 : new TextEncoder().encode(body).byteLength;
218+ settling =
219+ meterUsage(meter, res.headers.get("content-type") ?? "")
220+ .then(async ({ usage, unreadable }) => {
221+ const u = limited && (unreadable || !usage) ? unreadCharge(estimate, sent) : (usage ?? estimate);
222+ await project.chargeUsage(run.task, run.run, u, estimate);
223+ if (booked) await settleSpend(env, booked.org, run.personId, weighted(u) - booked.tokens, booked.month);
224+ })
225+ .catch((e) => console.error("reading model usage failed", e))
226+ .finally(free);
227+ return new Response(agentCopy, { status: res.status, headers: out });
228+ })().catch((e) => {
229+ free();
230+ throw e;
231+ });
232+ c.executionCtx.waitUntil(work.then(() => settling, () => undefined));
233+ return work;
149234 }
modifiedapps/diffinery/src/agentic/reviews.ts+16−8View file
@@ -10,7 +10,8 @@
1010 import type { FindingRow } from "./ProjectDO.js";
1111 import { asksForReview, REPO_DAILY_DEFAULT, REPO_DAILY_MAX, REVIEW_TOKENS_DEFAULT, REVIEW_TOKENS_MAX, REVIEW_TOKENS_MIN, REVIEW_APP, REVIEW_CHECK, REVIEW_CREATOR, TRIGGERS, AUTHORS, type ReviewAuthors, type ReviewSettings, type ReviewTrigger } from "./review.js";
1212 import { isRuntime, MAX_COMMAND, RUNTIMES, runtimeNeeds, type RuntimeId } from "./runtimes.js";
13-import { modelRoute, projectStub, providersFor, requirePersonSession } from "./tasks.js";
13+import { modelRoute, projectStub, requirePersonSession } from "./tasks.js";
14+import { payerFor, reviewRefusal, type Payer } from "./org-policy.js";
1415
1516 /**
1617 * Agent code review in the Worker: who may turn it on, who may ask for a
@@ -59,6 +60,8 @@
5960 if (current) await stub.setReviewSettings({ ...current, enabled: false, updatedAt: now() });
6061 return current ? { ...current, enabled: false } : null;
6162 }
63+ const refused = await reviewRefusal(c.env, l.repo);
64+ if (refused) throw new HttpError(403, refused);
6265 const runtime = typeof input.runtime === "string" ? input.runtime : "claude-code";
6366 if (!isRuntime(runtime)) throw new HttpError(422, "`runtime` must be claude-code, codex or custom");
6467 const command = runtime === "custom" ? (typeof input.command === "string" ? input.command.trim() : "") : null;
@@ -78,7 +81,7 @@
7881 throw e;
7982 }
8083 const route = modelRoute(runtime, input.provider, input.model);
81- await providersFor(c.env, me, runtime, route.via);
84+ await payerFor(c.env, c.get("dir"), l.repo, me, runtime, route.via);
8285 const dailyLimit = input.daily_limit === undefined || input.daily_limit === "" ? REPO_DAILY_DEFAULT : Number(input.daily_limit);
8386 if (!Number.isInteger(dailyLimit) || dailyLimit < 1 || dailyLimit > REPO_DAILY_MAX) throw new HttpError(422, `\`daily_limit\` must be a whole number from 1 to ${REPO_DAILY_MAX}`);
8487 const tokensPerReview = input.tokens_per_review === undefined || input.tokens_per_review === "" ? REVIEW_TOKENS_DEFAULT : Number(input.tokens_per_review);
@@ -132,12 +135,17 @@
132135 if (e instanceof TaskAccessError) return blocked(`${settings.person.login} can no longer run reviews as ${settings.agent.login} (${e.message}); an admin can set agent review up again`);
133136 throw e;
134137 }
135- const have = await new ModelKeys(env).providers(person.id);
136- const needs = runtimeNeeds(settings.runtime, settings.via ?? null);
137- const missing = needs.filter((p) => !have.has(p));
138- if (missing.length) return blocked(`${settings.person.login}'s ${missing.map((p) => PROVIDERS[p].name).join(" and ")} API key is gone; they can add it in Settings → Model access`);
139- const providers: Provider[] = settings.runtime === "custom" ? [...have] : needs;
140- const r = await projectStub(env, repo.id).requestReview({ repoId: repo.id, pr, head: issue.pull.head, base: issue.pull.base, headSha: head, settings, providers, requestedBy: requestedBy?.login ?? null });
138+ // The organization's policy (phase 5b): reviews allowed, and whose key pays, decided now for this review.
139+ const refused = await reviewRefusal(env, repo);
140+ if (refused) return blocked(refused);
141+ let payer: Payer;
142+ try {
143+ payer = await payerFor(env, dir, repo, person, settings.runtime, settings.via ?? null);
144+ } catch (e) {
145+ if (!(e instanceof HttpError)) throw e;
146+ return blocked(e.status === 422 && e.detail.includes("Settings → Model access") ? `${settings.person.login}'s ${runtimeNeeds(settings.runtime, settings.via ?? null).map((p) => PROVIDERS[p].name).join(" and ")} API key is gone; they can add it in Settings → Model access` : `${e.detail}; an admin can set agent review up again`);
147+ }
148+ const r = await projectStub(env, repo.id).requestReview({ repoId: repo.id, pr, head: issue.pull.head, base: issue.pull.base, headSha: head, settings, providers: payer.providers, keyOwner: payer.keyOwner, requestedBy: requestedBy?.login ?? null });
141149 return { status: r.status, task: r.task, ...(r.status === "skipped" ? { reason: "this head was already reviewed" } : r.reason ? { reason: r.reason } : {}) };
142150 }
143151
modifiedapps/diffinery/src/agentic/tasks-web.tsx+3−3View file
@@ -54,7 +54,7 @@
5454
5555 const RUN_KIND: Record<TaskView["runs"][number]["kind"], string> = { start: "Start", message: "Follow-up", retry: "Retry", review: "Review", coordinate: "Coordinate", plan: "Plan", build: "Build", fix: "Fix checks" };
5656
57-const UsageCells: FC<{ u: Omit<RunUsage, "source"> | null; subscription?: boolean }> = ({ u, subscription }) =>
57+export const UsageCells: FC<{ u: Omit<RunUsage, "source"> | null; subscription?: boolean }> = ({ u, subscription }) =>
5858 u ? (
5959 <>
6060 <td class="r">{u.requests ? u.requests.toLocaleString("en-US") : "—"}</td>
@@ -120,7 +120,7 @@
120120 </table>
121121 </div>
122122 <p class="muted small mt">
123- Requests through your key are counted by Diffinery from the provider's answers. A run in your terminal is counted by the CLI signed in there. Cost is only ever Claude Code's own figure: Codex and custom commands report none.
123+ {t.key_owner && t.key_owner !== t.person_id ? "The organization's key paid for this task's model requests. " : ""}Requests through a key are counted by Diffinery from the provider's answers. A run in your terminal is counted by the CLI signed in there. Cost is only ever Claude Code's own figure: Codex and custom commands report none.
124124 {subscription ? " * At API prices: a subscription covers it." : ""} Your usage across repositories is in <a href="/settings/model-access#usage">Settings → Model access</a>.
125125 </p>
126126 </div>
@@ -267,7 +267,7 @@
267267 <p class="muted">None of your agents' runs used a model in the last {report.days} days.</p>
268268 )}
269269 <p class="muted small mt">
270- Runs you pay for: your tasks and your coordinator's, and agent reviews in repositories where you turned them on. Repositories are named as they were when the runs ended. Diffinery counts requests through your keys from the provider's answers; runs in your terminal count what the CLI signed in there reported. Cost is only Claude Code's own figure, at API prices; check your provider's console for what you were billed. Also at <code>GET /api/v3/diffinery/model-usage</code>.
270+ Runs you pay for: your tasks and your coordinator's, and agent reviews in repositories where you turned them on. Runs an organization's key paid for are on that organization's Settings tab, for its owners. Repositories are named as they were when the runs ended. Diffinery counts requests through your keys from the provider's answers; runs in your terminal count what the CLI signed in there reported. Cost is only Claude Code's own figure, at API prices; check your provider's console for what you were billed. Also at <code>GET /api/v3/diffinery/model-usage</code>.
271271 </p>
272272 </div>
273273 </div>
modifiedapps/diffinery/src/agentic/tasks.ts+16−15View file
@@ -6,7 +6,8 @@
66 import { HttpError } from "../lib/util.js";
77 import { atLeast, type Caller } from "../repo/types.js";
88 import { mayDriveAgent, taskCaller, TaskAccessError } from "./access.js";
9-import { isProvider, ModelKeys, PROVIDERS, type Provider } from "./keys.js";
9+import { isProvider, type Provider } from "./keys.js";
10+import { payerFor } from "./org-policy.js";
1011 import type { AttemptView, TaskRow, TaskView } from "./ProjectDO.js";
1112 import { MAX_ATTEMPTS } from "./project.js";
1213 import { checksState, type ChecksState } from "./today-repo.js";
@@ -55,14 +56,12 @@
5556 return t;
5657 }
5758
58-/** The providers a run gets, checked against the runtime's needs (or the provider it goes through) and the person's keys. */
59-export async function providersFor(env: Env, person: Account, runtime: RuntimeId, via: Provider | null = null): Promise<Provider[]> {
60- const have = await new ModelKeys(env).providers(person.id);
61- const needs = runtimeNeeds(runtime, via);
62- for (const p of needs) {
63- if (!have.has(p)) throw new HttpError(422, `${runtimeLabel(runtime, { via })} needs your ${PROVIDERS[p].name} API key: add it in Settings → Model access`);
64- }
65- return runtime === "custom" ? [...have] : needs;
59+/**
60+ * The providers a run of `task` gets, and that whose key paid for it at its start (org-policy.ts) still may: a
61+ * message, retry or plan approval keeps the task's key owner.
62+ */
63+export async function taskProviders(c: AppContext, l: Loaded, person: Account, task: TaskRow): Promise<Provider[]> {
64+ return (await payerFor(c.env, c.get("dir"), l.repo, person, task.runtime, task.via, task.key_owner ?? task.person_id)).providers;
6665 }
6766
6867 /**
@@ -150,12 +149,12 @@
150149 for (const k of ["fix_checks", "land"] as const) if (input[k] !== undefined && typeof input[k] !== "boolean") throw new HttpError(422, `\`${k}\` must be true or false`);
151150 const fixChecks = input.fix_checks === undefined ? !pull : input.fix_checks === true;
152151 const land = input.land === true;
153- const providers = await providersFor(c.env, person, runtime, route.via);
152+ const { providers, keyOwner, orgMaxRunning } = await payerFor(c.env, c.get("dir"), l.repo, person, runtime, route.via);
154153 const project = projectStub(c.env, l.repo.id);
155154 const { planFirst: policy } = await project.projectSettings();
156155 const planFirst = policy === "required" || (input.plan_first === undefined ? policy === "default" : input.plan_first === true);
157- const t = await project.create({ repoId: l.repo.id, goal, runtime, command, route, agent: { id: agent.id, login: agent.login }, person: { id: person.id, login: person.login }, base, providers, pull, planFirst, attempts, fixChecks, land });
158- c.executionCtx.waitUntil(c.get("dir").audit({ actorId: person.id, actorLogin: person.login, action: "task.start", target: `${l.repo.owner_login}/${l.repo.name}#${t.number}`, detail: { agent: agent.login, runtime, ...(route.via ? { provider: route.via, model: route.model } : {}), ...(planFirst ? { plan_first: true } : {}), ...(attempts > 1 ? { attempts } : {}), fix_checks: fixChecks, land } }));
156+ const t = await project.create({ repoId: l.repo.id, goal, runtime, command, route, agent: { id: agent.id, login: agent.login }, person: { id: person.id, login: person.login }, base, providers, pull, planFirst, attempts, fixChecks, land, keyOwner, orgMaxRunning });
157+ c.executionCtx.waitUntil(c.get("dir").audit({ actorId: person.id, actorLogin: person.login, action: "task.start", target: `${l.repo.owner_login}/${l.repo.name}#${t.number}`, detail: { agent: agent.login, runtime, ...(keyOwner !== person.id ? { paid_by: l.repo.owner_login } : {}), ...(route.via ? { provider: route.via, model: route.model } : {}), ...(planFirst ? { plan_first: true } : {}), ...(attempts > 1 ? { attempts } : {}), fix_checks: fixChecks, land } }));
159158 return t;
160159 }
161160
@@ -178,7 +177,7 @@
178177 if (!body) throw new HttpError(422, "`body` is required");
179178 if (body.length > MAX_GOAL) throw new HttpError(422, `\`body\` is longer than ${MAX_GOAL} characters`);
180179 const { person, task } = await starter(c, l, number);
181- const t = await projectStub(c.env, l.repo.id).message(number, body, await providersFor(c.env, person, task.runtime, task.via));
180+ const t = await projectStub(c.env, l.repo.id).message(number, body, await taskProviders(c, l, person, task));
182181 c.executionCtx.waitUntil(c.get("dir").audit({ actorId: person.id, actorLogin: person.login, action: "task.message", target: `${l.repo.owner_login}/${l.repo.name}#${number}` }));
183182 return t;
184183 }
@@ -255,14 +254,14 @@
255254 /** Approve a task's waiting plan: its build run starts. */
256255 export async function approvePlan(c: AppContext, l: Loaded, number: number): Promise<TaskView> {
257256 const { person, task } = await starter(c, l, number);
258- const t = await projectStub(c.env, l.repo.id).approvePlan(number, await providersFor(c.env, person, task.runtime, task.via), person.login);
257+ const t = await projectStub(c.env, l.repo.id).approvePlan(number, await taskProviders(c, l, person, task), person.login);
259258 c.executionCtx.waitUntil(c.get("dir").audit({ actorId: person.id, actorLogin: person.login, action: "task.plan_approve", target: `${l.repo.owner_login}/${l.repo.name}#${number}` }));
260259 return t;
261260 }
262261
263262 export async function retryTask(c: AppContext, l: Loaded, number: number): Promise<TaskView> {
264263 const { person, task } = await starter(c, l, number);
265- const t = await projectStub(c.env, l.repo.id).retry(number, await providersFor(c.env, person, task.runtime, task.via));
264+ const t = await projectStub(c.env, l.repo.id).retry(number, await taskProviders(c, l, person, task));
266265 c.executionCtx.waitUntil(c.get("dir").audit({ actorId: person.id, actorLogin: person.login, action: "task.retry", target: `${l.repo.owner_login}/${l.repo.name}#${number}` }));
267266 return t;
268267 }
@@ -306,6 +305,8 @@
306305 model: t.model ?? null,
307306 agent: { login: t.agent_login, type: "Bot" },
308307 started_by: { login: t.person_login, type: "User" },
308+ // Phase 5b: whose key pays, fixed when the task started: the person's own, or the organization's.
309+ paid_by: t.key_owner && t.key_owner !== t.person_id ? "organization" : "person",
309310 plan_first: !!t.plan_first,
310311 ...(t.plan_first ? { plan_state: t.plan_state, plan: t.plan } : {}),
311312 ...(t.parent_task ? { parent_task: t.parent_task } : {}),
modifiedapps/diffinery/src/agentic/terminals.ts+9−3View file
@@ -9,6 +9,7 @@
99 import type { TaskView } from "./ProjectDO.js";
1010 import { MAX_GOAL, RUNTIME_VERSIONS, type RuntimeId } from "./runtimes.js";
1111 import { projectStub, requirePersonSession, starter } from "./tasks.js";
12+import { terminalRefusal } from "./org-policy.js";
1213
1314 /**
1415 * Web terminals (phase 4, /mnt/project-files/agentic/phase4-design.md): a sandbox one person opens
@@ -146,7 +147,10 @@
146147 const me = requirePersonSession(c);
147148 if (!terminalsEnabled(c.env)) throw new HttpError(403, "terminals are switched off on this Diffinery");
148149 if (l.repo.archived) throw new HttpError(403, "repository is archived (read-only)");
150+ // The organization's agent policy (phase 5b): terminals, and tasks in them, allowed.
149151 const dir = c.get("dir");
152+ const refused = await terminalRefusal(c.env, l.repo, { tasks: !!input.tasks, person: { dir, account: me } });
153+ if (refused) throw new HttpError(403, refused);
150154 const agent = input.agent ? await dir.accountByLogin(input.agent) : null;
151155 if (!agent || agent.kind !== "agent") throw new HttpError(422, "choose one of your agents");
152156 try {
@@ -214,8 +218,10 @@
214218 }
215219
216220 /** The person's own open terminal in this repository, with the opt-in in force, checked again at every dispatch. */
217-async function dispatchTerminal(c: AppContext, l: Loaded, me: Account, id: string | undefined, agentId?: string): Promise<{ t: TerminalRow; to: { id: string; sandboxId: string } }> {
221+async function dispatchTerminal(c: AppContext, l: Loaded, me: Account, id: string | undefined, runtime: RuntimeId, agentId?: string): Promise<{ t: TerminalRow; to: { id: string; sandboxId: string } }> {
218222 if (!terminalsEnabled(c.env)) throw new HttpError(403, "terminals are switched off on this Diffinery");
223+ const refused = await terminalRefusal(c.env, l.repo, { tasks: true, runtime, person: { dir: c.get("dir"), account: me } });
224+ if (refused) throw new HttpError(403, refused);
219225 const terminals = new Terminals(c.env);
220226 const t = await terminals.mine(terminalId(id), me, l.repo);
221227 if (t.status !== "running" || !t.sandbox_id) throw new HttpError(409, "the terminal isn't running");
@@ -247,7 +253,7 @@
247253 if (!goal) throw new HttpError(422, "write the goal");
248254 if (goal.length > MAX_GOAL) throw new HttpError(422, `the goal is longer than ${MAX_GOAL} characters`);
249255 const runtime = terminalRuntime(l, input.runtime);
250- const { t, to } = await dispatchTerminal(c, l, me, id);
256+ const { t, to } = await dispatchTerminal(c, l, me, id, runtime);
251257 const dir = c.get("dir");
252258 const agent = await dir.account(t.agent_id);
253259 if (!agent) throw new HttpError(403, "the terminal's agent no longer exists");
@@ -273,7 +279,7 @@
273279 const runtime = terminalRuntime(l, task.runtime);
274280 const text = typeof message === "string" ? message.trim() : "";
275281 if (text.length > MAX_GOAL) throw new HttpError(422, `the message is longer than ${MAX_GOAL} characters`);
276- const { t, to } = await dispatchTerminal(c, l, person, id, task.agent_id);
282+ const { t, to } = await dispatchTerminal(c, l, person, id, runtime, task.agent_id);
277283 const out = await projectStub(c.env, l.repo.id).runInTerminal(number, to, text || null);
278284 audit(c, person, l.repo, "terminal.task", { terminal: t.id, task: number, runtime, version: RUNTIME_VERSIONS[runtime as "claude-code" | "codex"] });
279285 return out;
modifiedapps/diffinery/src/agentic/usage.ts+126−28View file
@@ -63,11 +63,23 @@
6363
6464 /** Read a response body (a tee of what goes back to the agent) for the usage it reports; null when it reports none. */
6565 export async function readUsage(body: ReadableStream<Uint8Array>, contentType: string): Promise<Usage | null> {
66+ return (await meterUsage(body, contentType)).usage;
67+}
68+
69+/**
70+ * The usage a response reports, and whether part of it couldn't be read: an SSE line too long to keep, or a
71+ * JSON body too large to parse. Usage may have been in that part (a streamed Responses answer reports it only
72+ * in `response.completed`, which carries the whole answer), so runs held to their bookings charge the most
73+ * the request could have cost instead (`unreadCharge`).
74+ */
75+export async function meterUsage(body: ReadableStream<Uint8Array>, contentType: string): Promise<{ usage: Usage | null; unreadable: boolean }> {
6676 const sse = contentType.includes("text/event-stream");
6777 const reader = body.pipeThrough(new TextDecoderStream()).getReader();
6878 let usage: Usage | null = null;
6979 let buf = "";
7080 let size = 0;
81+ let dropping = false;
82+ let unreadable = false;
7183 try {
7284 for (;;) {
7385 const { done, value } = await reader.read();
@@ -82,17 +94,35 @@
8294 while ((nl = buf.indexOf("\n")) >= 0) {
8395 const line = buf.slice(0, nl).trim();
8496 buf = buf.slice(nl + 1);
97+ // The rest of a line already dropped.
98+ if (dropping) {
99+ dropping = false;
100+ continue;
101+ }
85102 if (line.startsWith("data:")) usage = fold(usage, usageOf(parse(line.slice(5).trim())));
86103 }
87- if (buf.length > MAX_SSE_LINE) buf = "";
104+ if (buf.length > MAX_SSE_LINE) {
105+ buf = "";
106+ dropping = true;
107+ unreadable = true;
108+ }
88109 }
89110 } finally {
90111 reader.releaseLock();
91112 }
92- if (sse) return buf.trim().startsWith("data:") ? fold(usage, usageOf(parse(buf.trim().slice(5).trim()))) : usage;
93- return size <= MAX_JSON ? fold(usage, usageOf(parse(buf))) : usage;
113+ if (sse) return { usage: !dropping && buf.trim().startsWith("data:") ? fold(usage, usageOf(parse(buf.trim().slice(5).trim()))) : usage, unreadable };
114+ if (size > MAX_JSON) return { usage, unreadable: true };
115+ return { usage: fold(usage, usageOf(parse(buf))), unreadable };
94116 }
95117
118+/**
119+ * What a request whose answer's usage couldn't be read, or that reported none, is charged on a run held to its
120+ * bookings: a token per byte it sent (three times its estimate's input; it can pull in nothing else, see
121+ * prepareRequest), and the output cap it was sent with. Images and PDFs bill by pixels and pages instead, which
122+ * the limit on requests at once (inflight.ts) bounds.
123+ */
124+export const unreadCharge = (estimate: Usage, size: number): Usage => ({ ...estimate, input: Math.max(estimate.input, size) });
125+
96126 export const formatTokens = (t: number): string => (t >= 1_000_000 ? `${+(t / 1_000_000).toFixed(1)}M` : t >= 1000 ? `${Math.round(t / 1000)}k` : String(t));
97127
98128 /**
@@ -184,7 +214,15 @@
184214 export const DEFAULT_MAX_OUTPUT = 32_000;
185215 /** What a compaction may write back, which its request can't cap: the most any current OpenAI model emits. */
186216 export const COMPACT_MAX_OUTPUT = 128_000;
217+/**
218+ * The most output one request books, whatever cap it asks for: more than any model emits, and small enough
219+ * that bookings and refunds add up exactly (a cap of 1e300 would swallow a month's count when refunded).
220+ */
221+export const MAX_OUTPUT_ESTIMATE = 1_000_000;
187222
223+/** Runs whose requests must stay within what was booked for them. */
224+const LIMITED = "agent reviews, runs with a token budget and runs on an organization's key";
225+
188226 /** Endpoints that generate tokens, so each request books an estimate, with the fields that cap their output. */
189227 const GENERATES: Record<string, string[]> = {
190228 "/v1/messages": ["max_tokens"],
@@ -202,7 +240,7 @@
202240 estimate: Usage | null;
203241 /** Why a review may not send this request, if it may not. */
204242 reviewRefuse?: string;
205- /** The body a review sends instead: every output cap set to the booked one, so the estimate holds. */
243+ /** The body a run held to its bookings sends instead: every output cap set to the booked one and no paid provider-side tools, so the estimate holds. */
206244 reviewBody?: string;
207245 /** The model the request asks for (null when it names none), for runs pinned to one. */
208246 model?: string | null;
@@ -215,15 +253,15 @@
215253 * plugins, web search, and more answers to pick from): a review or a run pinned to a model may not set them,
216254 * or its code could switch to a far pricier model than the one its budget was set for.
217255 */
218-const ROUTING = ["models", "route", "provider", "plugins", "web_search_options", "speed", "best_of"] as const;
256+const ROUTING = ["models", "route", "provider", "plugins", "web_search_options", "speed", "best_of", "preset"] as const;
219257
220258 /**
221259 * Why a run may not send a prepared request: a review never sets routing fields, and a run pinned to a model
222260 * (one that goes through OpenRouter with a model chosen in Diffinery) asks only for that model.
223261 */
224-export function pinRefusal(p: Pick<PreparedRequest, "estimate" | "model" | "routing">, run: { review: boolean; model: string | null }): string | undefined {
225- const who = run.review ? "agent reviews and other runs with a token budget" : "this task";
226- if ((run.review || run.model) && p.routing) return `${p.routing} isn't available to ${who}: Diffinery pins the model and its routing`;
262+export function pinRefusal(p: Pick<PreparedRequest, "estimate" | "model" | "routing">, run: { review: boolean; model: string | null; orgKey?: boolean }): string | undefined {
263+ const who = run.review || run.orgKey ? LIMITED : "this task";
264+ if ((run.review || run.orgKey || run.model) && p.routing) return `${p.routing} isn't available to ${who}: Diffinery pins the model and its routing`;
227265 if (run.model && p.estimate && p.model !== run.model) return `this ${run.review ? "review" : "task"} runs on ${run.model}${p.model ? `, not ${p.model}` : ""}`;
228266 return undefined;
229267 }
@@ -240,6 +278,9 @@
240278 const typeOf = (t: unknown): unknown => (t && typeof t === "object" ? (t as { type?: unknown }).type : undefined);
241279 const clientTool = (t: unknown): boolean => {
242280 const type = typeOf(t);
281+ // A tool with an environment other than the agent's own (OpenAI's hosted shell, in a provider's container) runs there.
282+ const env = t && typeof t === "object" ? (t as { environment?: unknown }).environment : undefined;
283+ if (env !== undefined && typeOf(env) !== "local") return false;
243284 return !!t && typeof t === "object" && (type === undefined || (typeof type === "string" && CLIENT_TOOL.test(type)));
244285 };
245286 /** The name a tool or a tool choice gives: Anthropic's and Responses' `name`, or Chat Completions' `function.name`. */
@@ -271,20 +312,66 @@
271312 return out;
272313 }
273314
274-/** Responses API fields that pull in context stored at the provider, which the request's size doesn't show. */
275-const STORED_CONTEXT = ["previous_response_id", "conversation", "prompt"] as const;
315+/**
316+ * Fields that pull in context stored at the provider, which the request's size doesn't show: the Responses API's
317+ * stored responses, conversations and prompts, and Anthropic's containers (with their skills and files).
318+ */
319+const storedContext = (path: string): readonly string[] => (path.startsWith("/v1/responses") ? ["previous_response_id", "conversation", "prompt"] : path === "/v1/messages" ? ["container"] : []);
276320
321+/** The fields of a request that carry its input (and none of its tools or settings). */
322+const INPUT_FIELDS = ["messages", "input", "system", "instructions"] as const;
323+
324+/** Keys whose value, a string or `{ url }`, is an image, file, audio or video given by URL unless it is a `data:` URL. */
325+const MEDIA_URL = ["image_url", "file_url", "video_url", "audio_url"] as const;
326+
277327 /**
328+ * Input the provider fetches or looks up itself, so the request's size doesn't show what it costs: Responses
329+ * `item_reference` items (stored context, like previous_response_id), anything given by file id, and images,
330+ * documents and files given by URL (Anthropic's `url` and `file` sources, Responses' `input_file` and `input_image`,
331+ * Chat Completions' `image_url` and `file` parts, OpenRouter's `file_data` URLs). Only what was sent counts:
332+ * a tool call's own arguments (Claude Code's WebFetch takes a URL) are never read as input; every other field
333+ * of the call is.
334+ */
335+export function remoteInput(x: Record<string, unknown>): string | undefined {
336+ const stack: unknown[] = INPUT_FIELDS.map((k) => x[k]);
337+ const isUrl = (v: unknown) => typeof v === "string" && !v.startsWith("data:");
338+ while (stack.length) {
339+ const v = stack.pop();
340+ if (!v || typeof v !== "object") continue;
341+ if (Array.isArray(v)) {
342+ for (const e of v) stack.push(e);
343+ continue;
344+ }
345+ const o = v as Record<string, unknown>;
346+ const type = o.type;
347+ if (type === "item_reference") return "item_reference inputs";
348+ if (typeof o.file_id === "string") return "inputs given by file id";
349+ const src = o.source as Record<string, unknown> | undefined;
350+ if (src && typeof src === "object" && (src.type === "url" || src.type === "file")) return `inputs given by ${src.type === "url" ? "URL" : "file id"}`;
351+ for (const k of MEDIA_URL) {
352+ const m = o[k];
353+ if (isUrl(m) || (m && typeof m === "object" && isUrl((m as { url?: unknown }).url))) return "inputs given by URL";
354+ }
355+ if (typeof o.file_data === "string" && /^[a-z][a-z0-9+.-]*:\/\//i.test(o.file_data)) return "inputs given by URL";
356+ // What a model asked a tool to do (its input or arguments) isn't input.
357+ const call = type === "tool_use" || type === "server_tool_use" || type === "function_call" || type === "custom_tool_call";
358+ for (const [k, e] of Object.entries(o)) if (e && typeof e === "object" && !(call && (k === "input" || k === "arguments"))) stack.push(e);
359+ }
360+ return undefined;
361+}
362+
363+/**
278364 * Look at a model request before it goes upstream, so that every answer can
279365 * be charged: its estimate (a token per 3 bytes of request, plus the output
280366 * it allows) is booked up front and settled to the usage the answer reports,
281- * and streamed Chat Completions are asked to include usage. A review may not
282- * start a background response, which it could leave unfetched and so
283- * uncharged, pull in stored context its estimate can't see, or ask for more
284- * than one answer; a review's request that doesn't cap its output is
285- * capped at the estimate's DEFAULT_MAX_OUTPUT, and paid provider-side tools
286- * are stripped from it. A compaction can't be capped,
287- * so it books COMPACT_MAX_OUTPUT.
367+ * and streamed Chat Completions are asked to include usage. A run held to
368+ * its bookings (LIMITED) may not start a background response, which it could
369+ * leave unfetched and so uncharged, pull in stored context or remote inputs
370+ * its estimate can't see, send a body that isn't a JSON object, or ask for
371+ * more than one answer; its request that doesn't cap its output is capped at
372+ * the estimate's DEFAULT_MAX_OUTPUT, and paid provider-side tools are
373+ * stripped from it. A compaction can't be capped, so it books
374+ * COMPACT_MAX_OUTPUT.
288375 */
289376 export function prepareRequest(method: string, path: string, body: string, size: number): PreparedRequest {
290377 const caps = method === "POST" && Object.hasOwn(GENERATES, path) ? GENERATES[path]! : null;
@@ -293,18 +380,26 @@
293380 const x = json && typeof json === "object" && !Array.isArray(json) ? (json as Record<string, unknown>) : null;
294381 // A request can set several caps (Chat Completions takes two): book the largest.
295382 const capped = x ? caps.map((k) => n(x[k])).filter((v) => v > 0) : [];
296- const cap = capped.length ? Math.max(...capped) : undefined;
383+ const cap = capped.length ? Math.min(Math.max(...capped), MAX_OUTPUT_ESTIMATE) : undefined;
297384 const estimate: Usage = { input: Math.ceil(size / 3), output: cap ?? (caps.length ? DEFAULT_MAX_OUTPUT : COMPACT_MAX_OUTPUT), cacheRead: 0, cacheWrite: 0 };
298- if (!x) return { body, estimate };
299- const stored = path.startsWith("/v1/responses") ? STORED_CONTEXT.find((k) => x[k] != null) : undefined;
385+ // A body that isn't a JSON object can't be capped or checked, and a lenient provider might still accept it.
386+ if (!x) return { body, estimate, reviewRefuse: `a request whose body isn't a JSON object isn't available to ${LIMITED}` };
387+ const stored = storedContext(path).find((k) => x[k] != null);
388+ const remote = remoteInput(x);
300389 const reviewRefuse =
301- x.background === true
302- ? "background responses aren't available to agent reviews and other runs with a token budget"
390+ x.background != null && x.background !== false
391+ ? `background responses aren't available to ${LIMITED}`
303392 : stored
304- ? `${stored} isn't available to agent reviews and other runs with a token budget: send the whole input instead`
305- : x.n != null && x.n !== 1
306- ? "more than one answer per request (n) isn't available to agent reviews and other runs with a token budget"
307- : undefined;
393+ ? `${stored} isn't available to ${LIMITED}: send the whole input instead`
394+ : remote
395+ ? `${remote} aren't available to ${LIMITED}: send the content itself instead`
396+ : x.n != null && x.n !== 1
397+ ? `more than one answer per request (n) isn't available to ${LIMITED}`
398+ : typeof x.model === "string" && /:online\b/i.test(x.model)
399+ ? `web search models (:online) aren't available to ${LIMITED}`
400+ : typeof x.model === "string" && /@preset\//i.test(x.model)
401+ ? `OpenRouter presets aren't available to ${LIMITED}`
402+ : undefined;
308403 let fields = x;
309404 if (path === "/v1/chat/completions" && x.stream === true) {
310405 const so = x.stream_options && typeof x.stream_options === "object" ? (x.stream_options as Record<string, unknown>) : {};
@@ -314,11 +409,14 @@
314409 if (reviewRefuse) out.reviewRefuse = reviewRefuse;
315410 const routing = ROUTING.find((k) => x[k] != null);
316411 if (routing) out.routing = routing;
317- // A review sends every cap as the one booked (the default when it sets none), and no paid provider-side tools.
412+ // A run held to its bookings sends every cap as the one booked (the default when it sets none), and no paid
413+ // provider-side tools: each of their steps reads more input than the request shows.
318414 const set = caps.filter((k) => x[k] !== undefined);
319415 const recap = caps[0] && (!set.length || set.some((k) => x[k] !== estimate.output));
320416 const unpaid = withoutPaidTools(fields);
321- if (recap || unpaid !== fields) out.reviewBody = JSON.stringify(recap ? { ...unpaid, ...Object.fromEntries((set.length ? set : [caps[0]!]).map((k) => [k, estimate.output])) } : unpaid);
417+ const capTo = (f: Record<string, unknown>) => ({ ...f, ...Object.fromEntries((set.length ? set : [caps[0]!]).map((k) => [k, estimate.output])) });
418+ // Always written out again, so a key the request repeats reaches the provider once, as read here.
419+ out.reviewBody = JSON.stringify(recap ? capTo(unpaid) : unpaid);
322420 return out;
323421 }
324422
modifiedapps/diffinery/src/http/api.ts+2−0View file
@@ -1,3 +1,4 @@
1+import { refreshRepoLimit } from "../agentic/org-policy.js";
12 import { Hono } from "hono";
23 import { fromUtf8 } from "@diffinery/git-core";
34 import type { Env } from "../env.js";
@@ -446,6 +447,7 @@
446447 const target = await dir.accountByLogin(b.new_owner.trim());
447448 if (!target) throw new HttpError(422, `no user or organization named ${b.new_owner.trim()}`, { errors: [{ resource: "Repository", code: "invalid", field: "new_owner" }] });
448449 const moved = await dir.transferRepo(repo, p.account, target, (b.new_name as string | null | undefined) ?? undefined);
450+ c.executionCtx.waitUntil(refreshRepoLimit(c.env, moved));
449451 auditRepo(c, caller, moved, "repo.transfer", { from: `${repo.owner_login}/${repo.name}`, to: `${moved.owner_login}/${moved.name}` });
450452 return c.json(repoOut(c, moved, (await dir.roleFor(p.account, moved)) ?? "read"), 202);
451453 });
modifiedapps/diffinery/src/lib/directory.ts+6−1View file
@@ -227,7 +227,7 @@
227227 /**
228228 * Delete an organization. Refused while it still owns repositories (transfer or delete them first),
229229 * container images or an import in progress. Its agents are disabled (their tokens revoked), its
230- * members, pending invitations, secrets and variables removed, and its login freed. The account row
230+ * members, pending invitations, secrets and variables, model keys and agent policy removed, and its login freed. The account row
231231 * stays, disabled under a placeholder login, because the audit log, import history and its agents
232232 * still refer to it. Repositories transferred out of it keep redirecting from their old URLs.
233233 */
@@ -248,6 +248,11 @@
248248 this.db.prepare("DELETE FROM org_secrets WHERE org_id = ?").bind(org.id),
249249 this.db.prepare("DELETE FROM org_secret_repos WHERE org_id = ?").bind(org.id),
250250 this.db.prepare("DELETE FROM org_variables WHERE org_id = ?").bind(org.id),
251+ // Phase 5b: its model keys, agent policy, allowance counts and the usage its keys paid for.
252+ this.db.prepare("DELETE FROM model_keys WHERE account_id = ?").bind(org.id),
253+ this.db.prepare("DELETE FROM org_agent_policies WHERE org_id = ?").bind(org.id),
254+ this.db.prepare("DELETE FROM org_key_spend WHERE org_id = ?").bind(org.id),
255+ this.db.prepare("DELETE FROM agent_usage WHERE key_owner = ?").bind(org.id),
251256 this.db.prepare("DELETE FROM login_redirects WHERE account_id = ?").bind(org.id),
252257 this.db.prepare("UPDATE accounts SET login = ?, status = 'disabled' WHERE id = ?").bind(`deleted-${org.id}`, org.id),
253258 ]);
modifiedapps/diffinery/src/mail/samples.ts+1−0View file
@@ -65,6 +65,7 @@
6565 item("token-expiring", "Access token expiring", { kind: "token_expiring", reason: "security_alert", data: { name: "ci-deploy", prefix: "dfy_Ab12Cd", expiresAt: exp } }),
6666 item("agent-created", "Agent added", { kind: "agent_created", reason: "security_alert", data: { agent: "review-bot" } }),
6767 item("model-key-added", "Model key added", { kind: "model_key_added", reason: "security_alert", data: { provider: "Anthropic", hint: "x7Qa" } }),
68+ item("org-model-key-added", "Organization model key added", { kind: "model_key_added", reason: "security_alert", data: { provider: "OpenAI", hint: "9fKe", org: "acme", by: me } }),
6869 item("deployment-review", "Deployment waiting for review", { kind: "deployment_review_requested", reason: "review_requested", data: { repo, environment: "production", workflow: "Deploy", runNumber: 128, title: "Release 2.4", branch: "main", sha: "4f1c2e9d0b7a6c5e3f2d1c0b9a8e7d6c5b4a3f2e", requestor: me, url: `${pub}/${repo}/actions/runs/sample` } }),
6970 item("review-limit", "Agent review limit reached", { kind: "review_limit_reached", reason: "your_activity", data: { repo, limit: 50, pr: 42 } }),
7071 item("push-bypassed", "Push protection bypassed", { kind: "push_bypassed", reason: "security_alert", data: { repo, pusher: me, kinds: ["AWS access key"], reason: "Fake key used by a test fixture", refs: ["refs/heads/main"] } }),
modifiedapps/diffinery/src/mail/templates.ts+21−4View file
@@ -447,15 +447,32 @@
447447 url: `${pub}/settings/agents`,
448448 };
449449 case "model_key_added": {
450+ const d = job.data;
451+ if (d.org) {
452+ const page = `${pub}/${d.org}?tab=settings#model-keys`;
453+ return {
454+ subject: `[Diffinery] An ${d.provider} API key was added to ${d.org}`,
455+ preheader: `Agents in ${d.org}'s repositories can now run on the organization's ${d.provider} key.`,
456+ eyebrow: { text: "Security", tone: "red" },
457+ title: [`An ${d.provider} API key was added to `, { b: d.org }],
458+ blocks: [
459+ { p: [`Agent runs that ${d.org}'s policy lets the organization pay for will use this key, and the provider bills the organization's account. Diffinery never shows the key again. You get this email because you're an owner of ${d.org}.`] },
460+ { kv: [["Organization", d.org], ["Provider", d.provider], ["Ends with", [{ code: `…${d.hint}` }]], ...(d.by ? ([["Added by", user(d.by)]] as Array<[string, string]>) : []), ["When", at(job.at)]] },
461+ { callout: ["If nobody in the organization added this key, delete it here and revoke it with the provider."], tone: "warn" },
462+ { button: { label: "Review the organization's model keys", url: page } },
463+ ],
464+ url: page,
465+ };
466+ }
450467 const page = `${pub}/settings/model-access`;
451468 return {
452- subject: `[Diffinery] Your ${job.data.provider} API key was added to Diffinery`,
453- preheader: `Agent tasks you start can now use your ${job.data.provider} key.`,
469+ subject: `[Diffinery] Your ${d.provider} API key was added to Diffinery`,
470+ preheader: `Agent tasks you start can now use your ${d.provider} key.`,
454471 eyebrow: { text: "Security", tone: "red" },
455- title: [`Your ${job.data.provider} API key was added to Diffinery`],
472+ title: [`Your ${d.provider} API key was added to Diffinery`],
456473 blocks: [
457474 { p: ["Agent tasks you start will use this key, and the provider bills your account for them. Diffinery never shows the key again."] },
458- { kv: [["Provider", job.data.provider], ["Ends with", [{ code: `…${job.data.hint}` }]], ["When", at(job.at)]] },
475+ { kv: [["Provider", d.provider], ["Ends with", [{ code: `…${d.hint}` }]], ["When", at(job.at)]] },
459476 { callout: ["If you didn't add this key, delete it here and revoke it with the provider."], tone: "warn" },
460477 { button: { label: "Review model access", url: page } },
461478 ],
modifiedapps/diffinery/src/mail/types.ts+1−1View file
@@ -77,7 +77,7 @@
7777 | { kind: "token_created"; data: { name: string; scopes: string[]; agent?: string } }
7878 | { kind: "token_expiring"; data: { name: string; prefix: string; expiresAt: string; agent?: string } }
7979 | { kind: "agent_created"; data: { agent: string } }
80- | { kind: "model_key_added"; data: { provider: string; hint: string } }
80+ | { kind: "model_key_added"; data: { provider: string; hint: string; /** An organization's key (phase 5b), and the owner who saved it. */ org?: string; by?: string } }
8181 | { kind: "review_limit_reached"; data: { repo: string; limit: number; pr: number } }
8282 | { kind: "deployment_review_requested"; data: { repo: string; environment: string; workflow: string; runNumber: number; title: string; branch: string; sha: string; requestor: string; url: string } }
8383 | { kind: "push_bypassed"; data: { repo: string; pusher: string; kinds: string[]; reason: string; refs?: string[] } }
modifiedapps/diffinery/src/web/account.tsx+3−0View file
@@ -2,6 +2,7 @@
22 import { raw } from "hono/html";
33 import type { Child, FC } from "hono/jsx";
44 import { requireAccount, requireRecentSignIn, type AppContext } from "../auth/principal.js";
5+import { OrgAgentSettings, orgAgentData } from "../agentic/org-web.js";
56 import { baseRole, type Account, type Directory, type Repo, type TokenScope } from "../lib/directory.js";
67 import { loadRepo } from "../http/common.js";
78 import { movedLoginPath } from "../lib/redirects.js";
@@ -2254,6 +2255,7 @@
22542255 </>
22552256 );
22562257 } else {
2258+ const agentData = await orgAgentData(c.env, acct.id);
22572259 body = (
22582260 <>
22592261 <h2>Settings</h2>
@@ -2277,6 +2279,7 @@
22772279 <button class="btn primary">Save</button>
22782280 </div>
22792281 </Form>
2282+ <OrgAgentSettings org={acct} data={agentData} t={t} />
22802283 <BlockList
22812284 blocks={await blockList(c.env, acct.id)}
22822285 add={`${base}/-/org/blocks`}
modifiedapps/diffinery/src/web/repo-settings.tsx+2−0View file
@@ -1,4 +1,5 @@
11 /** @jsxImportSource hono/jsx */
2+import { refreshRepoLimit } from "../agentic/org-policy.js";
23 import { raw } from "hono/html";
34 import type { Child, FC } from "hono/jsx";
45 import { requireAccount, requireRecentSignIn, type AppContext } from "../auth/principal.js";
@@ -957,6 +958,7 @@
957958 const target = await dir.accountByLogin(f.s("new_owner").trim());
958959 if (!target) throw new HttpError(422, "choose the new owner");
959960 const moved = await dir.transferRepo(l.repo, me, target, f.s("new_name").trim() || undefined);
961+ c.executionCtx.waitUntil(refreshRepoLimit(c.env, moved));
960962 audit(c, { ...l, repo: moved }, "repo.transfer", { from: full, to: `${moved.owner_login}/${moved.name}` });
961963 flash(c, "notice", `Transferred ${full} to ${moved.owner_login}/${moved.name}.`);
962964 return `${repoUrl(moved)}/settings`;
modifiedapps/diffinery/src/web/routes.tsx+2−0View file
@@ -5,6 +5,7 @@
55 import { registerAccount } from "./account.js";
66 import { registerAccountLifecycle } from "./account-lifecycle.js";
77 import { registerTasks } from "../agentic/tasks-web.js";
8+import { registerOrgAgents } from "../agentic/org-web.js";
89 import { registerReviews } from "../agentic/review-web.js";
910 import { registerProject } from "../agentic/project-web.js";
1011 import { registerTerminals } from "../agentic/terminal-web.js";
@@ -55,6 +56,7 @@
5556 registerImports(web);
5657 // Settings → Model access and /:owner/:repo/tasks: before the /:owner and /:owner/:repo catch-alls.
5758 registerTasks(web);
59+registerOrgAgents(web);
5860 registerReviews(web);
5961 registerProject(web);
6062 registerTerminals(web);
modifiedapps/diffinery/test/e2e/agents-harness.mjs+5−2View file
@@ -108,8 +108,11 @@
108108 req.on("data", (d) => (body += d));
109109 req.on("end", () => {
110110 modelRequests.push({ method: req.method, url: req.url, headers: req.headers, body });
111- res.writeHead(200, { "content-type": "application/json", "set-cookie": "upstream=1" });
112- res.end(JSON.stringify({ type: "message", content: [{ type: "text", text: "fake model says hi" }], usage: { input_tokens: 21, output_tokens: 4, cache_read_input_tokens: 300 } }));
111+ // A request that asks for it is answered after a while, so several can be on their way at once.
112+ setTimeout(() => {
113+ res.writeHead(200, { "content-type": "application/json", "set-cookie": "upstream=1" });
114+ res.end(JSON.stringify({ type: "message", content: [{ type: "text", text: "fake model says hi" }], usage: { input_tokens: 21, output_tokens: 4, cache_read_input_tokens: 300 } }));
115+ }, body.includes('"slow"') ? 3000 : 0);
113116 });
114117 });
115118 await new Promise((r) => upstream.listen(0, "127.0.0.1", r));
modifiedapps/diffinery/test/e2e/review.test.mjs+4−3View file
@@ -541,11 +541,12 @@
541541 // A streamed Chat Completion is asked for usage, and one with no output cap is capped at what its estimate assumed.
542542 assert.equal(JSON.parse(served3[1].body).stream_options.include_usage, true);
543543 assert.equal(JSON.parse(served3[1].body).max_completion_tokens, 32_000);
544- // 42 bytes / 3 + max_tokens 7 for the answer without usage, the streamed answer's own 40,000,
545- // and the estimate the cut-off answer was booked, which stays charged since its usage was never read.
544+ // A token per byte sent (42) + max_tokens 7 for the answer without usage (Security: no usage is charged like
545+ // unreadable usage), the streamed answer's own 40,000, and the estimate the cut-off answer was booked, which
546+ // stays charged since its usage was never read.
546547 const cut = Math.ceil(Buffer.byteLength(CUT) / 3) + 500;
547548 // The two-caps and the tools answers each report usage 10 in, 5 out.
548- await until("the estimates and the streamed usage to be counted", async () => (await runTokens(run3.number)) === 14 + 7 + 40_000 + 15 + 15 + cut, 15_000);
549+ await until("the estimates and the streamed usage to be counted", async () => (await runTokens(run3.number)) === 42 + 7 + 40_000 + 15 + 15 + cut, 15_000);
549550
550551 // Per repository: past the payer's daily limit, a request is refused, the check asks for action, and the payer gets one email.
551552 assert.equal((await asPerson("PUT", S, { ...settings, daily_limit: 1 })).status, 200);
modifiedapps/diffinery/test/e2e/tasks.test.mjs+237−0View file
@@ -976,6 +976,243 @@
976976 assert.ok((await ok("GET", P)).board.done.some((t) => t.number === n), "a landed task is done");
977977 });
978978
979+test("organizations: an agent policy, model keys only owners hold, and whose key pays (phase 5b)", async () => {
980+ const me = await ok("GET", "/user");
981+ const owner = me.login;
982+ const ORG = "acme-agents";
983+ const O = `/orgs/${ORG}/diffinery`;
984+ const ORG_KEY = `sk-ant-api03-${randomBytes(30).toString("hex")}`;
985+ const RIVAL_KEY = `sk-ant-api03-${randomBytes(30).toString("hex")}`;
986+ const rivalCookie = `__Host-dfy_session=${RIVAL_SESSION}`;
987+ const asRival = async (method, path, body) => {
988+ const res = await fetch(`${base}/api/v3${path}`, { method, headers: { cookie: rivalCookie, "content-type": "application/json" }, body: body === undefined ? undefined : JSON.stringify(body) });
989+ const text = await res.text();
990+ let json = null;
991+ try {
992+ json = text ? JSON.parse(text) : null;
993+ } catch {
994+ json = text;
995+ }
996+ return { status: res.status, json };
997+ };
998+ // A run's request through the model proxy, with its token, as the agent's CLI would send it.
999+ const proxyBody = (tok, body) => fetch(`${base}/-/model/anthropic/v1/messages`, { method: "POST", headers: { "x-api-key": tok, "content-type": "application/json" }, body });
1000+ const proxy = (tok, max = 100) => proxyBody(tok, JSON.stringify({ model: "m", max_tokens: max, messages: [] }));
1001+ // A web form, as the owner in their browser.
1002+ const form = async (page, action, fields) => {
1003+ const res = await fetch(`${base}${page}`);
1004+ const csrf = /name="_csrf" value="([^"]+)"/.exec(await res.text())?.[1];
1005+ const cookie = /dfy_csrf=([^;]+)/.exec(res.headers.get("set-cookie") ?? "")?.[1];
1006+ return fetch(`${base}${action}`, { method: "POST", redirect: "manual", headers: { cookie: `dfy_csrf=${cookie}`, "sec-fetch-site": "same-origin" }, body: new URLSearchParams([["_csrf", csrf ?? ""], ...fields]) });
1007+ };
1008+ const runToken = async (file) => {
1009+ await until(`the agent to write ${file}`, async () => existsSync(join(work, file)));
1010+ return readFileSync(join(work, file), "utf8");
1011+ };
1012+ const stopped = (R, n) =>
1013+ until(`task #${n} to stop`, async () => {
1014+ const t = await ok("GET", `${R}/${n}`);
1015+ return t.state !== "running" && t.state !== "queued" ? t : null;
1016+ });
1017+
1018+ await ok("POST", "/user/orgs", { login: ORG });
1019+ await ok("POST", `/orgs/${ORG}/repos`, { name: "org-tasks" });
1020+ const src = join(work, "org-src");
1021+ mkdirSync(src, { recursive: true });
1022+ writeFileSync(join(src, "README.md"), "# org tasks\n");
1023+ git(src, ["init", "-q", "-b", "main"]);
1024+ git(src, ["add", "-A"]);
1025+ git(src, ["commit", "-q", "-m", "initial"]);
1026+ git(src, ["remote", "add", "origin", `${base}/${ORG}/org-tasks.git`]);
1027+ git(src, ["push", "-q", "origin", "main"]);
1028+ assert.equal((await asPerson("PUT", "/diffinery/model-keys/anthropic", { key: ANTHROPIC_KEY, accept_terms: true })).status, 200);
1029+ assert.equal((await asPerson("POST", "/diffinery/agents", { login: "org-bot" })).status, 201);
1030+
1031+ // Nothing changes until an owner sets something; only owners see it, and never with a repository-limited token.
1032+ assert.deepEqual(await ok("GET", `${O}/agent-policy`), { keys: "personal", runtimes: ["claude-code", "codex", "custom"], providers: ["anthropic", "openai", "openrouter"], max_running: 10, terminals: true, terminal_tasks: true, review: true, member_monthly_tokens: null, updated_by: null, updated_at: null });
1033+ assert.equal((await asRival("GET", `${O}/agent-policy`)).status, 404, "not a member");
1034+ const limited = (await asPerson("POST", "/diffinery/tokens", { name: "org-limited", scopes: ["admin"], repositories: [`${ORG}/org-tasks`] })).json.token;
1035+ for (const path of ["agent-policy", "model-keys", "model-usage"]) assert.equal((await api("GET", `${O}/${path}`, undefined, limited)).status, 403, path);
1036+
1037+ // The organization's key: an owner in a browser, accepting the terms for it; never shown again.
1038+ const viaToken = await api("PUT", `${O}/model-keys/anthropic`, { key: ORG_KEY, accept_terms: true });
1039+ assert.equal(viaToken.status, 403, JSON.stringify(viaToken.json));
1040+ assert.equal((await asPerson("PUT", `${O}/model-keys/anthropic`, { key: ORG_KEY })).status, 422);
1041+ const saved = await asPerson("PUT", `${O}/model-keys/anthropic`, { key: ORG_KEY, accept_terms: true });
1042+ assert.equal(saved.status, 200, JSON.stringify(saved.json));
1043+ assert.equal(saved.json.hint, ORG_KEY.slice(-4));
1044+ assert.equal(saved.json.opted_in_by, owner);
1045+ const keys = await ok("GET", `${O}/model-keys`);
1046+ assert.deepEqual(keys.map((k) => k.provider), ["anthropic"]);
1047+ assert.ok(!JSON.stringify(keys).includes(ORG_KEY));
1048+ assert.ok(!(await ok("GET", "/diffinery/model-keys")).some((k) => k.hint === ORG_KEY.slice(-4)), "a person's keys are their own");
1049+
1050+ // The policy: owners in a browser.
1051+ assert.equal((await api("PUT", `${O}/agent-policy`, { keys: "organization" })).status, 403, "a token can't change it");
1052+ assert.equal((await asPerson("PUT", `${O}/agent-policy`, { max_running: 11 })).status, 422);
1053+ const policy = await asPerson("PUT", `${O}/agent-policy`, { keys: "organization", runtimes: ["claude-code", "custom"], max_running: 2, member_monthly_tokens: 10000 });
1054+ assert.equal(policy.status, 200, JSON.stringify(policy.json));
1055+ assert.equal(policy.json.updated_by, owner);
1056+ // Its repositories' "tasks at once" is held to it.
1057+ await until("the organization's limit to reach the project", async () => (await ok("GET", `/repos/${ORG}/org-tasks/diffinery/project`)).settings.org_max_running === 2);
1058+
1059+ const R = `/repos/${ORG}/org-tasks/diffinery/tasks`;
1060+ const codex = await asPerson("POST", R, { goal: "Not allowed", runtime: "codex", agent: "org-bot" });
1061+ assert.equal(codex.status, 403, JSON.stringify(codex.json));
1062+ assert.match(codex.json.message, /acme-agents's agent policy doesn't allow Codex/);
1063+
1064+ // An owner's task runs on the organization's key, which the provider gets and the sandbox never sees.
1065+ const started = await asPerson("POST", R, { goal: "On the organization's key", runtime: "custom", command: `printf %s "$ANTHROPIC_API_KEY" > ${join(work, "org-token")}; echo "key is $ANTHROPIC_API_KEY"; sleep 120`, agent: "org-bot" });
1066+ assert.equal(started.status, 201, JSON.stringify(started.json));
1067+ assert.equal(started.json.paid_by, "organization");
1068+ const n1 = started.json.number;
1069+ const t1 = await runToken("org-token");
1070+ assert.match(t1, /^dmt_/);
1071+ const sent = modelRequests.length;
1072+ const first = await proxy(t1);
1073+ assert.equal(first.status, 200, await first.clone().text());
1074+ assert.equal(modelRequests.length, sent + 1);
1075+ assert.equal(modelRequests.at(-1).headers["x-api-key"], ORG_KEY);
1076+ // The month's count settles to what the answer used: 21 + 4 + 300 cache reads at a tenth.
1077+ const month = await until("the month's count", async () => {
1078+ const u = await ok("GET", `${O}/model-usage`);
1079+ return u.month.people.find((p) => p.person === owner && p.tokens === 55) ? u : null;
1080+ });
1081+ assert.equal(month.month.member_monthly_tokens, 10000);
1082+ // A request that could take the person past the allowance never reaches the provider, and the task stops.
1083+ const over = await proxy(t1, 20000);
1084+ assert.equal(over.status, 403);
1085+ assert.match((await over.json()).error.message, /allowance of 10,000 tokens per person on acme-agents's keys is used up/);
1086+ assert.equal(modelRequests.length, sent + 1);
1087+ assert.equal((await stopped(R, n1)).state, "stopped");
1088+ const lines = (await ok("GET", `${R}/${n1}/transcript`)).map((l) => l.text).join("\n");
1089+ assert.match(lines, /Stopped: this month's allowance/);
1090+ assert.ok(!lines.includes(ORG_KEY));
1091+ // The run is the organization's to see, not in the owner's own 30 days.
1092+ const orgReport = await until("the organization's report", async () => {
1093+ const u = await ok("GET", `${O}/model-usage`);
1094+ return u.repositories.find((x) => x.repository === `${ORG}/org-tasks`) ? u : null;
1095+ });
1096+ assert.deepEqual(orgReport.repositories[0], { repository: `${ORG}/org-tasks`, kind: "task", runs: 1, reported_by_cli_runs: 0, requests: 1, input_tokens: 21, output_tokens: 4, cache_read_tokens: 300, cache_write_tokens: 0, cost_usd: null, person: owner });
1097+ assert.ok(!(await ok("GET", "/diffinery/model-usage")).repositories.some((x) => x.repository === `${ORG}/org-tasks`));
1098+
1099+ // Someone outside the organization, with write access, their own agent and their own key.
1100+ await ok("PUT", `/repos/${ORG}/org-tasks/collaborators/rival`, { permission: "push" });
1101+ const inv = (await asRival("GET", "/user/repository_invitations")).json.find((i) => i.repository.full_name === `${ORG}/org-tasks`);
1102+ assert.equal((await asRival("PATCH", `/user/repository_invitations/${inv.id}`)).status, 204);
1103+ assert.equal((await asRival("PUT", "/diffinery/model-keys/anthropic", { key: RIVAL_KEY, accept_terms: true })).status, 200);
1104+ assert.equal((await asRival("POST", "/diffinery/agents", { login: "rival-bot" })).status, 201);
1105+ const outside = await asRival("POST", R, { goal: "Outside", runtime: "custom", command: "true", agent: "rival-bot" });
1106+ assert.equal(outside.status, 403, JSON.stringify(outside.json));
1107+ assert.match(outside.json.message, /only its members can use/);
1108+ // Under `either`, they pay with their own key.
1109+ assert.equal((await asPerson("PUT", `${O}/agent-policy`, { keys: "either", member_monthly_tokens: null })).status, 200);
1110+ const own = await asRival("POST", R, { goal: "Own key", runtime: "custom", command: `printf %s "$ANTHROPIC_API_KEY" > ${join(work, "rival-token")}; sleep 120`, agent: "rival-bot" });
1111+ assert.equal(own.status, 201, JSON.stringify(own.json));
1112+ assert.equal(own.json.paid_by, "person");
1113+ const t2 = await runToken("rival-token");
1114+ assert.equal((await proxy(t2)).status, 200);
1115+ assert.equal(modelRequests.at(-1).headers["x-api-key"], RIVAL_KEY);
1116+ // On their own key, a request goes as they sent it.
1117+ assert.equal((await proxyBody(t2, '{"model":"m","background":true,"messages":[]}')).status, 200);
1118+ assert.deepEqual(JSON.parse(modelRequests.at(-1).body), { model: "m", background: true, messages: [] });
1119+ assert.equal((await asRival("POST", `${R}/${own.json.number}/stop`)).status, 200);
1120+
1121+ // Once a member, their task runs on the organization's key, until they're removed: the next request is refused.
1122+ const invited = await form(`/${ORG}?tab=people`, `/${ORG}/-/org/people`, [["who", "rival"], ["role", "member"]]);
1123+ assert.equal(invited.status, 303);
1124+ assert.equal((await asRival("PATCH", `/user/memberships/orgs/${ORG}`, { state: "active" })).status, 200);
1125+ const member = await asRival("POST", R, { goal: "As a member", runtime: "custom", command: `printf %s "$ANTHROPIC_API_KEY" > ${join(work, "member-token")}; sleep 120`, agent: "rival-bot" });
1126+ assert.equal(member.status, 201, JSON.stringify(member.json));
1127+ assert.equal(member.json.paid_by, "organization");
1128+ const t3 = await runToken("member-token");
1129+ assert.equal((await proxy(t3)).status, 200);
1130+ assert.equal(modelRequests.at(-1).headers["x-api-key"], ORG_KEY);
1131+ // On the organization's key, a run sends only what its booking bounds (Security M1): no background responses,
1132+ // stored context or several answers, and its output capped as booked.
1133+ const sentBefore = modelRequests.length;
1134+ const background = await proxyBody(t3, '{"model":"m","max_tokens":100,"background":true,"messages":[]}');
1135+ assert.equal(background.status, 403);
1136+ assert.match((await background.json()).error.message, /background responses aren't available to agent reviews, runs with a token budget and runs on an organization's key/);
1137+ assert.equal((await proxyBody(t3, '{"model":"m","max_tokens":100,"n":3,"messages":[]}')).status, 403);
1138+ // Nor input the request doesn't carry (Security's re-check): a PDF by URL, a stored item, a body that isn't JSON.
1139+ const byUrl = await proxyBody(t3, JSON.stringify({ model: "m", max_tokens: 16, messages: [{ role: "user", content: [{ type: "document", source: { type: "url", url: "https://example.com/big.pdf" } }] }] }));
1140+ assert.equal(byUrl.status, 403);
1141+ assert.match((await byUrl.json()).error.message, /^inputs given by URL aren't available to agent reviews, runs with a token budget and runs on an organization's key/);
1142+ assert.equal((await proxyBody(t3, '{"model":"m","max_tokens":16,"background":"true","messages":[]}')).status, 403);
1143+ assert.equal((await proxyBody(t3, '{"model":"m","max_tokens":16,"temperature":NaN,"messages":[]}')).status, 403);
1144+ assert.equal(modelRequests.length, sentBefore);
1145+ // Its tools that run on the provider are stripped; those the agent runs stay.
1146+ assert.equal((await proxyBody(t3, JSON.stringify({ model: "m", messages: [], tools: [{ name: "Read", input_schema: {} }, { type: "web_fetch_20250910", name: "web_fetch" }] }))).status, 200);
1147+ assert.deepEqual(JSON.parse(modelRequests.at(-1).body), { model: "m", messages: [], tools: [{ name: "Read", input_schema: {} }], max_tokens: 32000 });
1148+ // An absurd cap books a bounded estimate (Security S2): the month's count stays exact, here 3 answers of 55.
1149+ assert.equal((await proxyBody(t3, '{"model":"m","max_tokens":1e300,"messages":[]}')).status, 200);
1150+ assert.equal(JSON.parse(modelRequests.at(-1).body).max_tokens, 1000000);
1151+ await until("the member's exact count", async () => (await ok("GET", `${O}/model-usage`)).month.people.find((p) => p.person === "rival" && p.tokens === 165));
1152+ // Under an allowance, a person has at most 4 requests on the organization's keys on their way at once (Security's third
1153+ // look: images and PDFs bill past their booking, so this bounds the overshoot); a fifth is told to try again.
1154+ assert.equal((await asPerson("PUT", `${O}/agent-policy`, { member_monthly_tokens: 1_000_000 })).status, 200);
1155+ const slow = await Promise.all([1, 2, 3, 4, 5].map((i) => proxyBody(t3, JSON.stringify({ model: "m", max_tokens: 10, metadata: { user_id: "slow" }, messages: [{ role: "user", content: `slow ${i}` }] }))));
1156+ assert.deepEqual(slow.map((r) => r.status).sort(), [200, 200, 200, 200, 429]);
1157+ const fifth = slow.find((r) => r.status === 429);
1158+ assert.equal(fifth.headers.get("retry-after"), "5");
1159+ assert.match((await fifth.json()).error.message, /4 of your model requests on acme-agents's keys are already on their way/);
1160+ // Once they are answered, the slots are free again.
1161+ await until("the slots to free up", async () => (await proxy(t3)).status === 200);
1162+ // A request whose sandbox hangs up while the provider is still answering is settled anyway, and frees its slot
1163+ // (Security's fourth look): the next 4 at once all go through.
1164+ const hangUp = new AbortController();
1165+ const dropped = fetch(`${base}/-/model/anthropic/v1/messages`, { method: "POST", signal: hangUp.signal, headers: { "x-api-key": t3, "content-type": "application/json" }, body: JSON.stringify({ model: "m", max_tokens: 10, metadata: { user_id: "slow" }, messages: [] }) }).catch(() => null);
1166+ await sleep(1000);
1167+ hangUp.abort();
1168+ await dropped;
1169+ await sleep(4000);
1170+ const after4 = await Promise.all([1, 2, 3, 4].map(() => proxyBody(t3, JSON.stringify({ model: "m", max_tokens: 10, metadata: { user_id: "slow" }, messages: [] }))));
1171+ assert.deepEqual(after4.map((r) => r.status), [200, 200, 200, 200]);
1172+ assert.equal((await asPerson("PUT", `${O}/agent-policy`, { member_monthly_tokens: null })).status, 200);
1173+ const removed = await form(`/${ORG}?tab=people`, `/${ORG}/-/org/members`, [["login", "rival"], ["role", "remove"]]);
1174+ assert.equal(removed.status, 303);
1175+ // Removal also takes away their grants on its repositories; with write access given back as an outside
1176+ // collaborator, they can still run the task, but not on the organization's key.
1177+ await ok("PUT", `/repos/${ORG}/org-tasks/collaborators/rival`, { permission: "push" });
1178+ const again = (await asRival("GET", "/user/repository_invitations")).json.find((i) => i.repository.full_name === `${ORG}/org-tasks`);
1179+ if (again) assert.equal((await asRival("PATCH", `/user/repository_invitations/${again.id}`)).status, 204);
1180+ const before = modelRequests.length;
1181+ const refused = await proxy(t3);
1182+ assert.equal(refused.status, 403);
1183+ assert.match((await refused.json()).error.message, /rival is no longer a member of acme-agents, whose key this task uses/);
1184+ assert.equal(modelRequests.length, before);
1185+ assert.equal((await stopped(R, member.json.number)).state, "stopped");
1186+ // Retrying it isn't theirs to pay with the organization's key any more.
1187+ const retry = await asRival("POST", `${R}/${member.json.number}/retry`);
1188+ assert.equal(retry.status, 403, JSON.stringify(retry.json));
1189+ assert.match(retry.json.message, /only members of acme-agents can run agents on its keys/);
1190+
1191+ // The Settings tab: owners see the policy, the key's last four characters and the usage; nobody sees the key.
1192+ const page = await (await fetch(`${base}/${ORG}?tab=settings`)).text();
1193+ assert.match(page, /Agent policy/);
1194+ assert.ok(page.includes(`…${ORG_KEY.slice(-4)}`));
1195+ assert.ok(!page.includes(ORG_KEY));
1196+ assert.match(page, /Paid with the organization/);
1197+ assert.equal((await fetch(`${base}/${ORG}?tab=settings`, { headers: { cookie: rivalCookie } })).status, 403);
1198+ // Its form saves the policy (Kumo checkboxes send only what's ticked).
1199+ const savedForm = await form(`/${ORG}?tab=settings`, `/${ORG}/-/org/agent-policy`, [
1200+ ["keys", "personal"],
1201+ ["runtimes", "claude-code"],
1202+ ["runtimes", "custom"],
1203+ ["providers", "anthropic"],
1204+ ["max_running", "3"],
1205+ ["terminal_tasks", "1"],
1206+ ["review", "1"],
1207+ ["member_monthly_tokens", ""],
1208+ ]);
1209+ assert.equal(savedForm.status, 303);
1210+ assert.deepEqual(await ok("GET", `${O}/agent-policy`), { ...(await ok("GET", `${O}/agent-policy`)), keys: "personal", runtimes: ["claude-code", "custom"], providers: ["anthropic"], max_running: 3, terminals: false, terminal_tasks: true, review: true, member_monthly_tokens: null });
1211+ // Deleting the key.
1212+ assert.equal((await asPerson("DELETE", `${O}/model-keys/anthropic`)).status, 204);
1213+ assert.deepEqual(await ok("GET", `${O}/model-keys`), []);
1214+});
1215+
9791216 test("losing access to the agent stops the task", async () => {
9801217 const me = await ok("GET", "/user");
9811218 const owner = me.login;
addedapps/diffinery/test/unit/inflight.test.ts+22−0View file
@@ -1,0 +1,22 @@
1+import { describe, expect, it } from "vitest";
2+import { freeSlot, IN_FLIGHT_LEASE_MS, MAX_IN_FLIGHT, takeSlot } from "../../src/agentic/inflight.js";
3+import { testD1 } from "./stubs/d1.js";
4+
5+describe("model requests at once (phase 5b)", () => {
6+ it("gives a scope at most MAX_IN_FLIGHT slots, frees them, and lets an expired lease stop counting", async () => {
7+ const env = { DB: testD1() };
8+ const at = Date.parse("2026-10-14T09:00:00Z");
9+ const ids = [];
10+ for (let i = 0; i < MAX_IN_FLIGHT; i++) ids.push(await takeSlot(env, "org:acme:mia", at));
11+ expect(ids.every((x) => typeof x === "string")).toBe(true);
12+ expect(await takeSlot(env, "org:acme:mia", at)).toBeNull();
13+ // Other scopes have their own.
14+ expect(await takeSlot(env, "org:acme:otto", at)).toBeTypeOf("string");
15+ await freeSlot(env, ids[0]!);
16+ expect(await takeSlot(env, "org:acme:mia", at)).toBeTypeOf("string");
17+ expect(await takeSlot(env, "org:acme:mia", at)).toBeNull();
18+ // A lease whose request never settled (its Worker died) counts until it expires.
19+ expect(await takeSlot(env, "org:acme:mia", at + IN_FLIGHT_LEASE_MS - 1)).toBeNull();
20+ expect(await takeSlot(env, "org:acme:mia", at + IN_FLIGHT_LEASE_MS)).toBeTypeOf("string");
21+ });
22+});
addedapps/diffinery/test/unit/org-policy.test.ts+227−0View file
@@ -1,0 +1,227 @@
1+import { beforeEach, describe, expect, it } from "vitest";
2+import type { Env } from "../../src/env.js";
3+import { Directory, type Account, type Repo } from "../../src/lib/directory.js";
4+import { HttpError } from "../../src/lib/util.js";
5+import { bookSpend, DEFAULT_POLICY, orgSpend, parsePolicy, payerFor, policyRefusal, proxyPayer, reviewRefusal, savePolicy, settleSpend, spendMonth, terminalRefusal, type AgentPolicy } from "../../src/agentic/org-policy.js";
6+import { MAX_OUTPUT_ESTIMATE, prepareRequest, weighted } from "../../src/agentic/usage.js";
7+import { testD1 } from "./stubs/d1.js";
8+
9+const at = new Date().toISOString();
10+
11+function world() {
12+ const DB = testD1();
13+ const env = { DB } as unknown as Env;
14+ const dir = new Directory(env);
15+ const add = (id: string, kind: string, owner: string | null = null) => DB.sqlite.prepare("INSERT INTO accounts (id, login, kind, status, owner_id, created_at) VALUES (?, ?, ?, 'active', ?, ?)").run(id, id, kind, owner, at);
16+ const member = (org: string, user: string, role = "member") => DB.sqlite.prepare("INSERT INTO org_members (org_id, user_id, role, created_at) VALUES (?, ?, ?, ?)").run(org, user, role, at);
17+ const key = (account: string, provider: string) => DB.sqlite.prepare("INSERT INTO model_keys (account_id, provider, ciphertext, hint, opted_in_at, created_at, updated_at) VALUES (?, ?, 'x', 'abcd', ?, ?, ?)").run(account, provider, at, at, at);
18+ const policy = (org: string, p: Partial<AgentPolicy>) => DB.sqlite.prepare("INSERT OR REPLACE INTO org_agent_policies (org_id, json, updated_by, updated_at) VALUES (?, ?, 'owner', ?)").run(org, JSON.stringify({ ...DEFAULT_POLICY, ...p }), at);
19+ const repo = (owner: string, kind: "user" | "org") => ({ id: `${owner}-repo`, owner_id: owner, owner_login: owner, owner_kind: kind, name: "repo" }) as Repo;
20+ add("acme", "org");
21+ add("other-org", "org");
22+ add("mia", "user");
23+ add("otto", "user");
24+ add("bot", "agent", "acme");
25+ member("acme", "mia");
26+ // An agent account added to the organization never spends its key.
27+ member("acme", "bot");
28+ return { env, dir, DB, key, policy, member, repo, person: async (id: string) => (await dir.account(id)) as Account };
29+}
30+
31+const refusal = async (p: Promise<unknown>) => {
32+ try {
33+ await p;
34+ } catch (e) {
35+ if (e instanceof HttpError) return `${e.status} ${e.detail}`;
36+ throw e;
37+ }
38+ return "allowed";
39+};
40+
41+describe("organization agent policy (phase 5b)", () => {
42+ it("parses a policy, keeping what's left out and refusing what's out of range", () => {
43+ expect(parsePolicy({}, DEFAULT_POLICY)).toEqual(DEFAULT_POLICY);
44+ const p = parsePolicy({ keys: "organization", runtimes: ["codex", "claude-code"], providers: ["openai"], max_running: "4", terminals: false, terminal_tasks: false, review: false, member_monthly_tokens: "2000000" }, DEFAULT_POLICY);
45+ expect(p).toEqual({ keys: "organization", runtimes: ["claude-code", "codex"], providers: ["openai"], maxRunning: 4, terminals: false, terminalTasks: false, review: false, memberMonthlyTokens: 2_000_000 });
46+ expect(parsePolicy({ member_monthly_tokens: null }, p).memberMonthlyTokens).toBeNull();
47+ for (const bad of [{ keys: "shared" }, { runtimes: [] }, { runtimes: ["vim"] }, { providers: [] }, { max_running: 11 }, { max_running: 0 }, { terminals: "yes" }, { member_monthly_tokens: 5 }, { member_monthly_tokens: 1.5e10 }]) {
48+ expect(() => parsePolicy(bad, DEFAULT_POLICY)).toThrow(HttpError);
49+ }
50+ });
51+
52+ it("names what a policy refuses", () => {
53+ const p = { ...DEFAULT_POLICY, runtimes: ["claude-code" as const], providers: ["anthropic" as const] };
54+ expect(policyRefusal("acme", p, "codex", ["openai"])).toBe("acme's agent policy doesn't allow Codex");
55+ expect(policyRefusal("acme", p, "claude-code", ["openrouter"])).toBe("acme's agent policy doesn't allow OpenRouter");
56+ expect(policyRefusal("acme", p, "claude-code", ["anthropic"])).toBeNull();
57+ });
58+
59+ it("refuses terminals, terminal tasks and agent review as the policy says", async () => {
60+ const w = world();
61+ const repo = w.repo("acme", "org");
62+ expect(await terminalRefusal(w.env, repo, { tasks: true, runtime: "codex" })).toBeNull();
63+ expect(await reviewRefusal(w.env, repo)).toBeNull();
64+ w.policy("acme", { terminalTasks: false, runtimes: ["claude-code"], review: false });
65+ expect(await terminalRefusal(w.env, repo, {})).toBeNull();
66+ expect(await terminalRefusal(w.env, repo, { tasks: true })).toBe("acme's agent policy doesn't allow agent tasks in terminals");
67+ expect(await reviewRefusal(w.env, repo)).toBe("acme's agent policy doesn't allow agent review");
68+ w.policy("acme", { runtimes: ["claude-code"] });
69+ expect(await terminalRefusal(w.env, repo, { tasks: true, runtime: "codex" })).toBe("acme's agent policy doesn't allow Codex");
70+ w.policy("acme", { terminals: false });
71+ expect(await terminalRefusal(w.env, repo, {})).toBe("acme's agent policy doesn't allow web terminals");
72+ // A person's repository has no policy.
73+ expect(await terminalRefusal(w.env, w.repo("mia", "user"), { tasks: true })).toBeNull();
74+ });
75+
76+ it("holds a terminal's tasks to the allowed providers, and to members under `organization`", async () => {
77+ const w = world();
78+ const repo = w.repo("acme", "org");
79+ const as = async (id: string) => ({ dir: w.dir, account: await w.person(id) });
80+ // Claude Code in a terminal talks to Anthropic on the person's subscription, Codex to OpenAI.
81+ w.policy("acme", { providers: ["openai"] });
82+ expect(await terminalRefusal(w.env, repo, { tasks: true, runtime: "claude-code", person: await as("mia") })).toBe("acme's agent policy doesn't allow Anthropic");
83+ expect(await terminalRefusal(w.env, repo, { tasks: true, runtime: "codex", person: await as("mia") })).toBeNull();
84+ // Only members run agent tasks when the organization pays only with its own keys; a terminal without tasks stays open to all.
85+ w.policy("acme", { keys: "organization" });
86+ expect(await terminalRefusal(w.env, repo, { tasks: true, runtime: "claude-code", person: await as("otto") })).toBe("acme's agent policy lets only its members run agent tasks in its repositories");
87+ expect(await terminalRefusal(w.env, repo, { tasks: false, person: await as("otto") })).toBeNull();
88+ expect(await terminalRefusal(w.env, repo, { tasks: true, runtime: "claude-code", person: await as("mia") })).toBeNull();
89+ w.policy("acme", { keys: "either" });
90+ expect(await terminalRefusal(w.env, repo, { tasks: true, runtime: "claude-code", person: await as("otto") })).toBeNull();
91+ });
92+
93+ it("audits a site administrator's policy change as such", async () => {
94+ const w = world();
95+ w.DB.sqlite.prepare("INSERT INTO accounts (id, login, kind, status, site_admin, created_at) VALUES ('root', 'root', 'user', 'active', 1, ?)").run(at);
96+ w.member("acme", "otto", "owner");
97+ const org = await w.person("acme");
98+ const detail = () => JSON.parse((w.DB.sqlite.prepare("SELECT detail FROM audit_log WHERE action = 'org.agent_policy.update' ORDER BY rowid DESC LIMIT 1").get() as { detail: string }).detail);
99+ const pending: Array<Promise<unknown>> = [];
100+ const wait = (p: Promise<unknown>) => void pending.push(p.catch(() => undefined));
101+ await savePolicy(w.env, w.dir, org, await w.person("otto"), DEFAULT_POLICY, wait);
102+ await Promise.all(pending);
103+ expect(detail().site_admin).toBeUndefined();
104+ await savePolicy(w.env, w.dir, org, await w.person("root"), DEFAULT_POLICY, wait);
105+ await Promise.all(pending);
106+ expect(detail().site_admin).toBe(true);
107+ });
108+
109+ describe("whose key pays when work starts", () => {
110+ let w: ReturnType<typeof world>;
111+ beforeEach(() => {
112+ w = world();
113+ w.key("mia", "anthropic");
114+ w.key("otto", "anthropic");
115+ w.key("acme", "anthropic");
116+ });
117+
118+ it("is the person's own in a person's repository, and in an organization without a policy", async () => {
119+ const mia = await w.person("mia");
120+ expect(await payerFor(w.env, w.dir, w.repo("mia", "user"), mia, "claude-code", null)).toMatchObject({ keyOwner: "mia", providers: ["anthropic"], orgMaxRunning: null });
121+ expect(await payerFor(w.env, w.dir, w.repo("acme", "org"), mia, "claude-code", null)).toMatchObject({ keyOwner: "mia", orgMaxRunning: 10 });
122+ expect(await refusal(payerFor(w.env, w.dir, w.repo("acme", "org"), mia, "codex", null))).toBe("422 Codex needs your OpenAI API key: add it in Settings → Model access");
123+ });
124+
125+ it("under `either`, is the organization's for members and the person's own for outside collaborators", async () => {
126+ w.policy("acme", { keys: "either", maxRunning: 2 });
127+ expect(await payerFor(w.env, w.dir, w.repo("acme", "org"), await w.person("mia"), "claude-code", null)).toEqual({ keyOwner: "acme", providers: ["anthropic"], orgMaxRunning: 2 });
128+ expect(await payerFor(w.env, w.dir, w.repo("acme", "org"), await w.person("otto"), "claude-code", null)).toMatchObject({ keyOwner: "otto" });
129+ // No organization key for the provider: the member's own.
130+ w.key("mia", "openai");
131+ expect(await payerFor(w.env, w.dir, w.repo("acme", "org"), await w.person("mia"), "codex", null)).toMatchObject({ keyOwner: "mia", providers: ["openai"] });
132+ });
133+
134+ it("under `organization`, refuses outside collaborators and agent accounts, and says when the organization lacks the key", async () => {
135+ w.policy("acme", { keys: "organization" });
136+ expect(await refusal(payerFor(w.env, w.dir, w.repo("acme", "org"), await w.person("otto"), "claude-code", null))).toBe("403 acme pays for agents in its repositories with its own keys, which only its members can use");
137+ expect(await refusal(payerFor(w.env, w.dir, w.repo("acme", "org"), await w.person("bot"), "claude-code", null))).toMatch(/^403 /);
138+ w.key("mia", "openai");
139+ expect(await refusal(payerFor(w.env, w.dir, w.repo("acme", "org"), await w.person("mia"), "codex", null))).toBe("422 acme pays for agents with its own keys, and has no OpenAI key: an owner can add it on the organization's Settings tab");
140+ });
141+
142+ it("refuses runtimes and providers the policy doesn't allow, and gives a custom command only allowed providers' keys", async () => {
143+ w.policy("acme", { keys: "either", runtimes: ["claude-code", "custom"], providers: ["anthropic"] });
144+ w.key("acme", "openai");
145+ const mia = await w.person("mia");
146+ expect(await refusal(payerFor(w.env, w.dir, w.repo("acme", "org"), mia, "codex", null))).toBe("403 acme's agent policy doesn't allow Codex");
147+ expect(await refusal(payerFor(w.env, w.dir, w.repo("acme", "org"), mia, "claude-code", "openrouter"))).toBe("403 acme's agent policy doesn't allow OpenRouter");
148+ expect(await payerFor(w.env, w.dir, w.repo("acme", "org"), mia, "custom", null)).toMatchObject({ keyOwner: "acme", providers: ["anthropic"] });
149+ });
150+
151+ it("keeps a task's key owner from its start, and refuses a follow-up that may no longer pay that way", async () => {
152+ w.policy("acme", { keys: "either" });
153+ const mia = await w.person("mia");
154+ const repo = w.repo("acme", "org");
155+ expect(await payerFor(w.env, w.dir, repo, mia, "claude-code", null, "mia")).toMatchObject({ keyOwner: "mia" });
156+ expect(await payerFor(w.env, w.dir, repo, mia, "claude-code", null, "acme")).toMatchObject({ keyOwner: "acme" });
157+ // A retry by someone outside the organization of a task the organization paid for.
158+ expect(await refusal(payerFor(w.env, w.dir, repo, await w.person("otto"), "claude-code", null, "acme"))).toBe("403 only members of acme can run agents on its keys");
159+ // Transferred to another organization, or to a person.
160+ expect(await refusal(payerFor(w.env, w.dir, w.repo("other-org", "org"), mia, "claude-code", null, "acme"))).toMatch(/^403 this task was paid for by an organization that no longer owns/);
161+ expect(await refusal(payerFor(w.env, w.dir, w.repo("mia", "user"), mia, "claude-code", null, "acme"))).toMatch(/^403 this task was paid for by an organization that no longer owns/);
162+ w.policy("acme", { keys: "organization" });
163+ expect(await refusal(payerFor(w.env, w.dir, repo, mia, "claude-code", null, "mia"))).toBe("403 acme now pays for agents in its repositories only with its own keys: start a new task");
164+ w.policy("acme", { keys: "personal" });
165+ expect(await refusal(payerFor(w.env, w.dir, repo, mia, "claude-code", null, "acme"))).toBe("403 acme no longer pays for agents with its keys: start a new task with yours");
166+ });
167+ });
168+
169+ describe("the model proxy's check on every request", () => {
170+ it("lets a member spend the organization's key, with the allowance, until something changes", async () => {
171+ const w = world();
172+ w.policy("acme", { keys: "either", memberMonthlyTokens: 50_000 });
173+ const repo = w.repo("acme", "org");
174+ const run = { repo, personId: "mia", keyOwner: "acme", provider: "anthropic" as const };
175+ expect(await proxyPayer(w.env, w.dir, run)).toEqual({ org: { id: "acme", login: "acme", cap: 50_000 } });
176+ expect(await proxyPayer(w.env, w.dir, { ...run, provider: "openai" })).toEqual({ org: { id: "acme", login: "acme", cap: 50_000 } });
177+ // An agent account that is "in" the organization is never a member who pays.
178+ expect(await proxyPayer(w.env, w.dir, { ...run, personId: "bot" })).toEqual({ refuse: "bot is no longer a member of acme, whose key this task uses" });
179+ // The repository moved to another organization mid-task.
180+ expect(await proxyPayer(w.env, w.dir, { ...run, repo: w.repo("other-org", "org") })).toEqual({ refuse: "the organization whose key this task uses no longer owns other-org/repo" });
181+ // The member was removed: the next request is refused, nothing cached.
182+ w.DB.sqlite.prepare("DELETE FROM org_members WHERE org_id = 'acme' AND user_id = 'mia'").run();
183+ expect(await proxyPayer(w.env, w.dir, run)).toEqual({ refuse: "mia is no longer a member of acme, whose key this task uses" });
184+ w.member("acme", "mia");
185+ w.policy("acme", { keys: "either", providers: ["openai"] });
186+ expect(await proxyPayer(w.env, w.dir, run)).toEqual({ refuse: "acme's agent policy doesn't allow Anthropic" });
187+ w.policy("acme", { keys: "personal" });
188+ expect(await proxyPayer(w.env, w.dir, run)).toEqual({ refuse: "acme no longer pays for agents with its keys" });
189+ });
190+
191+ it("lets a personal key pay unless the organization now pays only with its own", async () => {
192+ const w = world();
193+ const run = { repo: w.repo("acme", "org"), personId: "mia", keyOwner: "mia", provider: "anthropic" as const };
194+ expect(await proxyPayer(w.env, w.dir, run)).toEqual({ org: null });
195+ expect(await proxyPayer(w.env, w.dir, { ...run, repo: w.repo("mia", "user") })).toEqual({ org: null });
196+ w.policy("acme", { keys: "organization" });
197+ expect(await proxyPayer(w.env, w.dir, run)).toEqual({ refuse: "acme now pays for agents in its repositories only with its own keys" });
198+ });
199+ });
200+
201+ it("books each request against the monthly allowance in one step, and settles it to what was used", async () => {
202+ const w = world();
203+ expect(await bookSpend(w.env, "acme", "mia", 600, 1000)).toBe(true);
204+ expect(await bookSpend(w.env, "acme", "mia", 500, 1000)).toBe(false);
205+ await settleSpend(w.env, "acme", "mia", -100);
206+ expect(await bookSpend(w.env, "acme", "mia", 500, 1000)).toBe(true);
207+ // At the allowance, even a request with no estimate is refused.
208+ expect(await bookSpend(w.env, "acme", "mia", 0, 1000)).toBe(false);
209+ // Another person, and a first request larger than the whole allowance.
210+ expect(await bookSpend(w.env, "acme", "otto", 2000, 1000)).toBe(false);
211+ expect(await bookSpend(w.env, "acme", "otto", 10, 1000)).toBe(true);
212+ // Without an allowance it always books, so a limit set later counts the month so far.
213+ expect(await bookSpend(w.env, "other-org", "mia", 5_000_000, null)).toBe(true);
214+ await settleSpend(w.env, "acme", "otto", -50);
215+ expect((await orgSpend(w.env, "acme")).map((x) => [x.login, x.tokens])).toEqual([
216+ ["mia", 1000],
217+ ["otto", 0],
218+ ]);
219+ expect(spendMonth(new Date("2026-10-31T23:59:59Z"))).toBe("2026-10");
220+ // A request asking for absurd output books a bounded estimate, so refunding it leaves the month's count exact.
221+ const huge = weighted(prepareRequest("POST", "/v1/messages", '{"max_tokens":1e300,"messages":[]}', 30).estimate!);
222+ expect(huge).toBe(10 + MAX_OUTPUT_ESTIMATE);
223+ expect(await bookSpend(w.env, "other-org", "mia", huge, null)).toBe(true);
224+ await settleSpend(w.env, "other-org", "mia", -huge);
225+ expect((await orgSpend(w.env, "other-org")).map((x) => x.tokens)).toEqual([5_000_000]);
226+ });
227+});
modifiedapps/diffinery/test/unit/usage.test.ts+81−7View file
@@ -1,5 +1,5 @@
11 import { describe, expect, it } from "vitest";
2-import { COMPACT_MAX_OUTPUT, DEFAULT_MAX_OUTPUT, formatCost, formatTokens, pinRefusal, prepareRequest, readBody, readUsage, reportedUsage, runUsage, totalUsage, usageJson, usageOf, weighted } from "../../src/agentic/usage.js";
2+import { COMPACT_MAX_OUTPUT, DEFAULT_MAX_OUTPUT, MAX_OUTPUT_ESTIMATE, formatCost, formatTokens, meterUsage, pinRefusal, prepareRequest, readBody, readUsage, remoteInput, reportedUsage, runUsage, totalUsage, unreadCharge, usageJson, usageOf, weighted } from "../../src/agentic/usage.js";
33
44 const stream = (chunks: string[]) =>
55 new ReadableStream<Uint8Array>({
@@ -46,13 +46,24 @@
4646 const huge = "data: " + "x".repeat(1100 * 1024);
4747 expect(await readUsage(stream([huge.slice(0, 600_000), huge.slice(600_000), '\ndata: {"usage":{"input_tokens":4,"output_tokens":2}}\n\n']), "text/event-stream")).toEqual({ input: 4, output: 2, cacheRead: 0, cacheWrite: 0 });
4848 });
49+ it("says when part of an answer was too large to read for usage", async () => {
50+ // A streamed Responses answer whose response.completed (the whole answer, with its usage) is over a megabyte.
51+ const completed = 'data: {"type":"response.completed","response":{"output":"' + "x".repeat(1100 * 1024) + '","usage":{"input_tokens":300000,"output_tokens":9}}}';
52+ const chunks = ['data: {"type":"response.created","response":{}}\n\n', completed.slice(0, 700_000), completed.slice(700_000, 1_400_000), completed.slice(1_400_000) + "\n\n"];
53+ expect(await meterUsage(stream(chunks), "text/event-stream")).toEqual({ usage: null, unreadable: true });
54+ // The tail of the dropped line isn't read as a line of its own.
55+ expect(await meterUsage(stream([completed.slice(0, 1_100_000), 'x"}}\ndata: {"usage":{"input_tokens":1,"output_tokens":1}}\n']), "text/event-stream")).toEqual({ usage: { input: 1, output: 1, cacheRead: 0, cacheWrite: 0 }, unreadable: true });
56+ expect(await meterUsage(stream(['data: {"usage":{"input_tokens":4,"output_tokens":2}}\n']), "text/event-stream")).toEqual({ usage: { input: 4, output: 2, cacheRead: 0, cacheWrite: 0 }, unreadable: false });
57+ // Such an answer is charged a token per byte sent, plus its output cap.
58+ expect(unreadCharge({ input: 100, output: 16, cacheRead: 0, cacheWrite: 0 }, 300)).toEqual({ input: 300, output: 16, cacheRead: 0, cacheWrite: 0 });
59+ });
4960 });
5061
5162 describe("prepareRequest", () => {
5263 const prep = (method: string, path: string, body: string) => prepareRequest(method, path, body, new TextEncoder().encode(body).length);
5364 it("estimates the most an answer may cost: a token per 3 bytes plus the output it allows", () => {
5465 const body = JSON.stringify({ model: "m", max_tokens: 1000, messages: [{ role: "user", content: "é".repeat(300) }] });
55- expect(prep("POST", "/v1/messages", body)).toEqual({ body, model: "m", estimate: { input: Math.ceil((body.length + 300) / 3), output: 1000, cacheRead: 0, cacheWrite: 0 } });
66+ expect(prep("POST", "/v1/messages", body)).toEqual({ body, reviewBody: body, model: "m", estimate: { input: Math.ceil((body.length + 300) / 3), output: 1000, cacheRead: 0, cacheWrite: 0 } });
5667 expect(prep("POST", "/v1/responses", '{"max_output_tokens":50}').estimate?.output).toBe(50);
5768 expect(prep("POST", "/v1/chat/completions", '{"max_completion_tokens":70}').estimate?.output).toBe(70);
5869 expect(prep("POST", "/v1/messages", '{"model":"m"}').estimate?.output).toBe(DEFAULT_MAX_OUTPUT);
@@ -64,7 +75,7 @@
6475 expect(prep("POST", "/v1/completions", '{"prompt":"x","max_tokens":80}').estimate?.output).toBe(80);
6576 // A compaction can't be capped: it books the most a model emits.
6677 expect(prep("POST", "/v1/responses/compact", '{"input":[],"max_output_tokens":5}').estimate?.output).toBe(COMPACT_MAX_OUTPUT);
67- expect(prep("POST", "/v1/responses/compact", '{"input":[]}').reviewBody).toBeUndefined();
78+ expect(prep("POST", "/v1/responses/compact", '{"input":[]}').reviewBody).toBe('{"input":[]}');
6879 // Requests that generate nothing are never estimated.
6980 expect(prep("POST", "/v1/messages/count_tokens", "{}").estimate).toBeNull();
7081 expect(prep("GET", "/v1/models", "").estimate).toBeNull();
@@ -91,7 +102,7 @@
91102 const p = prep("POST", "/v1/chat/completions", '{"max_completion_tokens":1,"max_tokens":100000}');
92103 expect(p.estimate?.output).toBe(100_000);
93104 expect(JSON.parse(p.reviewBody!)).toEqual({ max_completion_tokens: 100_000, max_tokens: 100_000 });
94- expect(prep("POST", "/v1/chat/completions", '{"max_completion_tokens":7,"max_tokens":7}').reviewBody).toBeUndefined();
105+ expect(JSON.parse(prep("POST", "/v1/chat/completions", '{"max_completion_tokens":7,"max_tokens":7}').reviewBody!)).toEqual({ max_completion_tokens: 7, max_tokens: 7 });
95106 });
96107 it("pins a review's or a pinned run's model and routing", () => {
97108 const req = (o: object) => prep("POST", "/v1/chat/completions", JSON.stringify({ max_tokens: 5, ...o }));
@@ -133,7 +144,7 @@
133144 // With no tools left, neither tools nor a tool choice is sent.
134145 expect(JSON.parse(prep("POST", "/v1/responses", JSON.stringify({ max_output_tokens: 9, tools: [{ type: "web_search" }], tool_choice: "required" })).reviewBody!)).toEqual({ max_output_tokens: 9 });
135146 // Nothing to strip and nothing to cap: the request goes as it is.
136- expect(prep("POST", "/v1/messages", JSON.stringify({ max_tokens: 9, tools: [fn] })).reviewBody).toBeUndefined();
147+ expect(JSON.parse(prep("POST", "/v1/messages", JSON.stringify({ max_tokens: 9, tools: [fn] })).reviewBody!)).toEqual({ max_tokens: 9, tools: [fn] });
137148 // Ordinary tasks keep their tools: only the review body changes.
138149 expect(JSON.parse(a.body).tools).toHaveLength(5);
139150 });
@@ -143,13 +154,76 @@
143154 const streamed = JSON.parse(prep("POST", "/v1/chat/completions", '{"stream":true}').reviewBody!);
144155 expect(streamed).toEqual({ stream: true, stream_options: { include_usage: true }, max_completion_tokens: DEFAULT_MAX_OUTPUT });
145156 // A request that caps its output, any way the endpoint takes it, is sent as it is.
146- expect(prep("POST", "/v1/chat/completions", '{"max_tokens":9}').reviewBody).toBeUndefined();
147- expect(prep("POST", "/v1/responses", '{"max_output_tokens":9}').reviewBody).toBeUndefined();
157+ // Written out again even when nothing changes, so a repeated key reaches the provider once, as it was read.
158+ expect(prep("POST", "/v1/chat/completions", '{"max_tokens":9}').reviewBody).toBe('{"max_tokens":9}');
159+ expect(prep("POST", "/v1/responses", '{"max_output_tokens":9,"model":"a","model":"b"}').reviewBody).toBe('{"max_output_tokens":9,"model":"b"}');
148160 expect(prep("POST", "/v1/messages/count_tokens", "{}").reviewBody).toBeUndefined();
149161 expect(JSON.parse(prep("POST", "/v1/completions", '{"prompt":"x"}').reviewBody!)).toEqual({ prompt: "x", max_tokens: DEFAULT_MAX_OUTPUT });
150162 // OpenRouter's text completions take a prompt, which isn't stored context there.
151163 expect(prep("POST", "/v1/completions", '{"prompt":"x","max_tokens":5}').reviewRefuse).toBeUndefined();
152164 });
165+ it("books and sends at most MAX_OUTPUT_ESTIMATE of output, whatever cap a request asks for (phase 5b)", () => {
166+ const huge = prep("POST", "/v1/messages", '{"max_tokens":1e300,"messages":[]}');
167+ expect(huge.estimate?.output).toBe(MAX_OUTPUT_ESTIMATE);
168+ expect(JSON.parse(huge.reviewBody!).max_tokens).toBe(MAX_OUTPUT_ESTIMATE);
169+ expect(prep("POST", "/v1/responses", '{"max_output_tokens":1e309}').estimate?.output).toBe(DEFAULT_MAX_OUTPUT);
170+ });
171+ it("refuses input a request doesn't carry, which its booking can't see (phase 5b)", () => {
172+ const refuse = (path: string, x: unknown) => prep("POST", path, JSON.stringify(x)).reviewRefuse;
173+ const msg = (content: unknown) => ({ max_tokens: 16, messages: [{ role: "user", content }] });
174+ // Stored context: Responses item references, and Anthropic containers.
175+ expect(refuse("/v1/responses", { input: [{ type: "item_reference", id: "msg_1" }] })).toMatch(/^item_reference inputs aren't available to agent reviews/);
176+ expect(refuse("/v1/messages", { ...msg("hi"), container: "container_1" })).toMatch(/^container isn't available/);
177+ // Anthropic documents and images by URL or file, Responses files and images by id or URL, Chat Completions parts by URL.
178+ expect(refuse("/v1/messages", msg([{ type: "document", source: { type: "url", url: "https://example.com/a.pdf" } }]))).toMatch(/^inputs given by URL aren't available/);
179+ expect(refuse("/v1/messages", msg([{ type: "image", source: { type: "file", file_id: "file_1" } }]))).toMatch(/^inputs given by file id/);
180+ expect(refuse("/v1/messages", { ...msg("x"), system: [{ type: "text", text: "s" }, { type: "document", source: { type: "url", url: "https://e/x" } }] })).toMatch(/URL/);
181+ expect(refuse("/v1/responses", { input: [{ role: "user", content: [{ type: "input_file", file_id: "file_1" }] }] })).toMatch(/file id/);
182+ expect(refuse("/v1/responses", { input: [{ role: "user", content: [{ type: "input_file", file_url: "https://e/a.pdf" }] }] })).toMatch(/URL/);
183+ expect(refuse("/v1/responses", { input: [{ role: "user", content: [{ type: "input_image", image_url: "https://e/a.png" }] }] })).toMatch(/URL/);
184+ expect(refuse("/v1/chat/completions", { messages: [{ role: "user", content: [{ type: "image_url", image_url: { url: "https://e/a.png" } }] }] })).toMatch(/URL/);
185+ expect(refuse("/v1/chat/completions", { messages: [{ role: "user", content: [{ type: "file", file: { filename: "a.pdf", file_data: "https://e/a.pdf" } }] }] })).toMatch(/URL/);
186+ // A tool result can't bring one in either, deep in the conversation.
187+ expect(refuse("/v1/messages", msg([{ type: "tool_result", tool_use_id: "t", content: [{ type: "image", source: { type: "url", url: "https://e/i.png" } }] }]))).toMatch(/URL/);
188+ // Content sent as data is fine, and so is a URL in a tool call's own arguments (Claude Code's WebFetch).
189+ expect(refuse("/v1/messages", msg([{ type: "image", source: { type: "base64", media_type: "image/png", data: "iVBO" } }, { type: "document", source: { type: "text", data: "t" } }]))).toBeUndefined();
190+ expect(refuse("/v1/responses", { input: [{ role: "user", content: [{ type: "input_image", image_url: "data:image/png;base64,iVBO" }, { type: "input_file", filename: "a.pdf", file_data: "data:application/pdf;base64,JVBE" }] }] })).toBeUndefined();
191+ expect(refuse("/v1/messages", { max_tokens: 9, messages: [{ role: "assistant", content: [{ type: "tool_use", id: "t", name: "WebFetch", input: { url: "https://e/x", file_id: "f" } }] }] })).toBeUndefined();
192+ expect(refuse("/v1/responses", { input: [{ type: "function_call", call_id: "c", name: "f", arguments: '{"image_url":"https://e"}' }] })).toBeUndefined();
193+ // Tools' schemas aren't input.
194+ expect(refuse("/v1/messages", { ...msg("x"), tools: [{ name: "fetch", input_schema: { type: "object", properties: { url: { type: "string" }, file_id: { type: "string" } } } }] })).toBeUndefined();
195+ expect(remoteInput({ messages: "plain text" })).toBeUndefined();
196+ // A call's type doesn't hide what else the object carries: only its input or arguments are skipped.
197+ expect(refuse("/v1/chat/completions", { messages: [{ role: "user", content: [{ type: "file", file: { type: "tool_use", file_data: "https://e/big.pdf" } }] }] })).toMatch(/URL/);
198+ expect(refuse("/v1/chat/completions", { messages: [{ role: "user", content: [{ type: "file", file: { type: "function_call", file_id: "file_1" } }] }] })).toMatch(/file id/);
199+ expect(refuse("/v1/messages", msg([{ type: "tool_use", id: "t", name: "x", input: {}, content: [{ type: "image", source: { type: "url", url: "https://e" } }] }]))).toMatch(/URL/);
200+ });
201+ it("refuses any background but false, bodies that aren't JSON objects, and :online models (phase 5b)", () => {
202+ for (const b of [true, "true", 1, {}]) expect(prep("POST", "/v1/responses", JSON.stringify({ background: b })).reviewRefuse).toMatch(/^background responses/);
203+ for (const b of [false, null]) expect(prep("POST", "/v1/responses", JSON.stringify({ background: b, input: "x" })).reviewRefuse).toBeUndefined();
204+ for (const body of ['{"temperature":NaN}', "[]", '"x"', ""]) expect(prep("POST", "/v1/messages", body).reviewRefuse).toMatch(/^a request whose body isn't a JSON object/);
205+ expect(prep("POST", "/v1/chat/completions", '{"model":"openai/gpt-4o:online","messages":[]}').reviewRefuse).toMatch(/^web search models \(:online\)/);
206+ expect(prep("POST", "/v1/chat/completions", '{"model":"openai/gpt-4o:online:free","messages":[]}').reviewRefuse).toMatch(/^web search models/);
207+ expect(prep("POST", "/v1/chat/completions", '{"model":"@preset/cheap","messages":[]}').reviewRefuse).toMatch(/^OpenRouter presets/);
208+ expect(prep("POST", "/v1/chat/completions", '{"model":"openai/gpt-4o@Preset/cheap","messages":[]}').reviewRefuse).toMatch(/^OpenRouter presets/);
209+ expect(prep("POST", "/v1/chat/completions", '{"model":"m","preset":"cheap","messages":[]}').routing).toBe("preset");
210+ // Requests that generate nothing aren't held to anything.
211+ expect(prep("POST", "/v1/messages/count_tokens", "not json").reviewRefuse).toBeUndefined();
212+ });
213+ it("strips paid provider-side tools from runs on an organization's key, as from reviews (phase 5b)", () => {
214+ const fn = { name: "Read", input_schema: {} };
215+ const p = prep("POST", "/v1/messages", JSON.stringify({ max_tokens: 9, messages: [], tools: [fn, { type: "web_fetch_20250910", name: "web_fetch" }], mcp_servers: [{ type: "url", url: "https://m" }] }));
216+ expect(JSON.parse(p.reviewBody!)).toEqual({ max_tokens: 9, messages: [], tools: [fn] });
217+ // A shell runs on the agent's side only with no environment, or a local one: OpenAI's hosted shell runs in a provider container.
218+ const shells = [{ type: "shell" }, { type: "shell", environment: { type: "local" } }, { type: "shell", environment: { type: "container_auto" } }];
219+ expect(JSON.parse(prep("POST", "/v1/responses", JSON.stringify({ max_output_tokens: 9, tools: shells })).reviewBody!).tools).toEqual(shells.slice(0, 2));
220+ });
221+ it("names runs on an organization's key among those held to their bookings", () => {
222+ expect(prep("POST", "/v1/responses", '{"background":true}').reviewRefuse).toBe("background responses aren't available to agent reviews, runs with a token budget and runs on an organization's key");
223+ const req = (x: Record<string, unknown>) => prep("POST", "/v1/chat/completions", JSON.stringify(x));
224+ expect(pinRefusal(req({ model: "m", models: ["a", "b"] }), { review: false, model: null, orgKey: true })).toMatch(/^models isn't available to agent reviews, runs with a token budget and runs on an organization's key/);
225+ expect(pinRefusal(req({ model: "m", models: ["a", "b"] }), { review: false, model: null })).toBeUndefined();
226+ });
153227 });
154228
155229 describe("readBody", () => {