Diffinery

codician/diffinerypublic

Open-source, agent-first GitHub and GitHub Actions alternative that runs on your own Cloudflare account: git hosting, pull requests, code review and drop-in GitHub Actions YAML on Workers, Durable Objects, R2 and Containers.

MASTER_KEY can be rotated: secrets carry a key id, and links signed with the previous key keep working (#206)

* Secrets carry a key id so MASTER_KEY can be rotated
* Links and tokens signed with MASTER_KEY_PREV keep working during a rotation
* Document how to rotate MASTER_KEY
test-bot authored 1 parent b0d9e57commit 5e6977eddb663669360988c25180b830e0519787Browse files
21 files changed+572−119
Files (21)
modifiedapps/diffinery/docs/security/overview.md+2−1View file
@@ -230,7 +230,8 @@
230230 1. The instance has one master key, `MASTER_KEY`, a base64-encoded secret configured on the Worker.
231231 2. For each scope (a repository, `repo:<id>`, an environment, `repo:<id>:env:<name>`, or an organization, `org:<id>`), Diffinery derives a separate 256-bit AES-GCM key from the master key with HKDF-SHA-256.
232232 3. Each secret value is encrypted with its scope's key and a random 96-bit IV. The scope is also bound in as additional authenticated data, so a ciphertext copied from one repository into another fails to decrypt.
233-4. Only the ciphertext is stored. Values are decrypted inside the repository's Durable Object only when a workflow run needs them.
233+4. Only the ciphertext is stored, with the ID of the master key that encrypted it. Values are decrypted inside the repository's Durable Object only when a workflow run needs them.
234+5. The master key can be rotated: while the replaced key is configured as `MASTER_KEY_PREV`, secrets encrypted with it still decrypt, and everything new is encrypted with the new key. See [Rotating MASTER_KEY](/docs/self-hosting/configuration#rotating-master_key).
234235
235236 Secret values are never returned by the API or shown in the web UI after they are saved; you can only replace or delete them. Only accounts with the admin role (organization owners and admins for organization secrets) can manage secrets, and agents can neither set nor delete them. Without `MASTER_KEY`, secrets cannot be saved. See [Secrets and variables](/docs/actions/secrets-and-variables) and, for operators, [Configuration](/docs/self-hosting/configuration).
236237
modifiedapps/diffinery/docs/self-hosting/configuration.md+37−2View file
@@ -159,13 +159,14 @@
159159
160160 | Secret | Format | Created by | What it does | If it is missing |
161161 | --- | --- | --- | --- | --- |
162-| `MASTER_KEY` | 32 random bytes, base64 | `ensure-secrets.sh`, once | Root key for envelope encryption of Actions secrets: each repository, organization and deployment environment gets its own AES-GCM key derived from it with HKDF. The GitHub token of a running import and each webhook's signing secret are encrypted the same way. It also signs the unsubscribe links in emails, the codes and OAuth state for linking GitHub accounts, keys the stored hashes of sign-up codes, and a key derived from it (HKDF) signs each job's `ACTIONS_RUNTIME_TOKEN` and the upload and download URLs of the Actions cache and artifact services. | Repository, organization and environment secrets can't be stored or read (`could not store the secret: ... (is MASTER_KEY configured?)`). Notification emails that carry unsubscribe links can't be built, so they end up in the email log as failed; security and account emails still send. Jobs get no `ACTIONS_RUNTIME_TOKEN`: `actions/cache` and the built-in cache step skip caching (`The cache service is not available to this job; skipping.`), and uploading or downloading artifacts fails (`The artifact service is not available to this job.`). Imports that come with a GitHub token can't be started. Webhooks can't be created or delivered. Open sign-up can't send codes. |
162+| `MASTER_KEY` | 32 random bytes, base64 | `ensure-secrets.sh`, once | Root key for envelope encryption of Actions secrets: each repository, organization and deployment environment gets its own AES-GCM key derived from it with HKDF. The GitHub token of a running import and each webhook's signing secret are encrypted the same way. Model provider keys are encrypted the same way. It also signs the unsubscribe links in emails, the links in invitation and email verification emails, the codes and OAuth state for linking GitHub accounts, keys the stored hashes of sign-up codes, and a key derived from it (HKDF) signs each job's `ACTIONS_RUNTIME_TOKEN` and the upload and download URLs of the Actions cache and artifact services. | Repository, organization and environment secrets can't be stored or read (`could not store the secret: ... (is MASTER_KEY configured?)`). Notification emails that carry unsubscribe links can't be built, so they end up in the email log as failed; security and account emails still send. Jobs get no `ACTIONS_RUNTIME_TOKEN`: `actions/cache` and the built-in cache step skip caching (`The cache service is not available to this job; skipping.`), and uploading or downloading artifacts fails (`The artifact service is not available to this job.`). Imports that come with a GitHub token can't be started. Webhooks can't be created or delivered. Open sign-up can't send codes. |
163+| `MASTER_KEY_PREV` | The `MASTER_KEY` you replaced | You, only while [rotating `MASTER_KEY`](#rotating-master_key) | Everything sealed or signed with the old key keeps working: secrets encrypted with it still decrypt, and links and tokens signed with it still verify. Nothing new uses it. | The normal state. Right after a rotation without it, secrets stored under the old key can't be read and links signed with it stop working, as if the old key had been lost. |
163164 | `OIDC_PRIVATE_KEY` | An ECDSA P-256 private key as a JWK (JSON), with a `kid` | `ensure-secrets.sh`, once | Signs the ID tokens that workflow jobs request with `permissions: id-token: write` (ES256). Its public half is served at `/.well-known/jwks`. | Jobs that request an ID token fail with `OIDC is not configured on this Diffinery instance`, `/.well-known/jwks` returns no keys, and `GET /api/v3/meta` returns `"oidc": false`. |
164165 | `GITHUB_OAUTH_CLIENT_SECRET` | The client secret of a GitHub OAuth app | You, optionally | With `GITHUB_OAUTH_CLIENT_ID`, turns on the **Link with GitHub** option on **Settings → Linked accounts**. | People link GitHub accounts with the gist code only. |
165166 | `MAIL_EVENTS_SECRET` | A random string, at least 32 characters | You, optionally | Turns on `POST /-/email/events`, the signed intake for bounces and complaints from anything other than Cloudflare's own event subscriptions. See [Bounces, complaints and the suppression list](/docs/admin/site-administration#bounces-complaints-and-the-suppression-list). | The endpoint answers 404. Bounces and complaints still arrive through the `diffinery-mail-events` queue. |
166167 | `TURNSTILE_SECRET` | The secret key of a Cloudflare Turnstile widget | The deploy, from its widget (see [Turnstile](#turnstile)) | With `TURNSTILE_SITE_KEY`, checks that whoever asks for a sign-up code (with `SIGNUP=open`), or sends a security report without signing in, passed the Turnstile widget. See [SIGNUP](#signup) and [Security contact](#security-contact). | With `SIGNUP=open`, sign-up answers `503` "Sign-up needs Turnstile configured", and the security report form takes reports only from people who are signed in (local development skips the check). The next deploy sets it when the token has **Account → Turnstile → Edit**. |
167168
168-None of these secrets is rotated automatically. Changing `MASTER_KEY` makes every stored secret undecryptable and invalidates unsubscribe links already sent, along with the runtime tokens of running jobs and cache and artifact links already handed out; keep a copy somewhere safe. To rotate the OIDC key, delete `OIDC_PRIVATE_KEY` and deploy: a new key with a new `kid` (`diffinery-<date>`) is generated, and relying parties pick it up from `/.well-known/jwks`.
169+None of these secrets is rotated automatically, and `ensure-secrets.sh` never replaces one that exists. Keep a copy of `MASTER_KEY` somewhere safe: replacing it without keeping the old one as `MASTER_KEY_PREV` makes every stored secret undecryptable and invalidates unsubscribe links already sent, along with the runtime tokens of running jobs and cache and artifact links already handed out. To rotate it, follow [Rotating MASTER_KEY](#rotating-master_key). To rotate the OIDC key, delete `OIDC_PRIVATE_KEY` and deploy: a new key with a new `kid` (`diffinery-<date>`) is generated, and relying parties pick it up from `/.well-known/jwks`.
169170
170171 To set a secret by hand (for example to restore a backup):
171172
@@ -174,6 +175,40 @@
174175 printf '%s' "$MASTER_KEY" | pnpm exec wrangler secret put MASTER_KEY -c wrangler.deploy.jsonc
175176 ```
176177
178+### Rotating MASTER_KEY
179+
180+Rotate `MASTER_KEY` when it may have leaked, or on your own schedule. Diffinery accepts two keys while a rotation is under way: `MASTER_KEY`, which encrypts and signs everything new, and `MASTER_KEY_PREV`, the key it replaced, which is only used to decrypt and verify what was made before.
181+
182+1. Make a new key and keep it with your copy of the current one:
183+
184+ ```sh
185+ NEW_KEY="$(openssl rand -base64 32)"
186+ ```
187+
188+2. Set the current key as `MASTER_KEY_PREV` and the new one as `MASTER_KEY` in one step, so no version of the Worker runs with the new key alone. `OLD_KEY` is the current value from your safe copy (Cloudflare doesn't show a secret's value):
189+
190+ ```sh
191+ cd apps/diffinery
192+ jq -n --arg old "$OLD_KEY" --arg new "$NEW_KEY" '{MASTER_KEY_PREV: $old, MASTER_KEY: $new}' | pnpm exec wrangler secret bulk -c wrangler.deploy.jsonc
193+ ```
194+
195+ To use `wrangler secret put` instead, put `MASTER_KEY_PREV` first and `MASTER_KEY` second: each command deploys a new version of the Worker, and a version with the new `MASTER_KEY` but no `MASTER_KEY_PREV` can't read the secrets stored so far.
196+
197+3. Save the new key where you kept the old one. Nothing else needs to change: no deploy, migration or restart.
198+
199+From then on:
200+
201+- New and replaced secrets (repository, organization and environment secrets, model provider keys, webhook secrets, import tokens) are encrypted with the new key. Each stored value names the key it was encrypted with (`v2.<key id>.…`, where the key id is the first 8 hex characters of the key's SHA-256: `printf %s "$NEW_KEY" | base64 -d | sha256sum | cut -c1-8`). Values stored before key ids existed (`v1.…`) are tried with both keys.
202+- Webhook secrets are re-encrypted with the new key the next time the webhook delivers, and the GitHub token of an import when the import is retried. Repository, organization and environment secrets and model provider keys stay encrypted with the old key until someone saves them again.
203+- Unsubscribe links, invitation and verification emails, sign-up codes, GitHub linking codes and OAuth state, runtime tokens, and cache and artifact URLs signed with the old key keep working; new ones are signed with the new key.
204+
205+Remove `MASTER_KEY_PREV` (`pnpm exec wrangler secret delete MASTER_KEY_PREV -c wrangler.deploy.jsonc`) only when nothing you still need depends on the old key:
206+
207+- Every repository, organization and environment secret and every model provider key has been saved again since the rotation. Their values can't be read back, so whoever manages each one re-enters it. One left behind can't be decrypted once `MASTER_KEY_PREV` is gone, and runs that use it fail.
208+- Links already sent have expired, or you accept that they stop working: runtime tokens and cache and artifact URLs last about as long as their job, sign-up codes 10 minutes, verification links 24 hours, invitation emails until the invitation expires, and unsubscribe links up to a year. Someone with an old unsubscribe link can still change their email settings after signing in.
209+
210+If the old key leaked, remove `MASTER_KEY_PREV` as soon as the secrets have been saved again: until then, anyone holding the old key can still forge what it signs, and every secret encrypted with it is only as safe as that key.
211+
177212 ## Bindings
178213
179214 Defined in `wrangler.jsonc`. The deploy creates the underlying resources if they don't exist.
modifiedapps/diffinery/docs/self-hosting/deploy.md+2−2View file
@@ -185,7 +185,7 @@
185185
186186 It also sets `TURNSTILE_SECRET` when the deployed config has a `TURNSTILE_SITE_KEY` and the Worker has no `TURNSTILE_SECRET` yet (the first deploy with Turnstile): `scripts/turnstile.sh secret <site key>` reads that widget's secret key from the Cloudflare API. Without **Account → Turnstile → Edit** it warns and the deploy goes on.
187187
188-Values are generated on the runner, or read from the Cloudflare API, and piped straight into `wrangler secret put`; they are never printed. The script lists existing secrets first and stops with `unexpected output from wrangler secret list` if it can't read them, rather than risk overwriting `MASTER_KEY`, which would make every stored Actions secret unreadable.
188+Values are generated on the runner, or read from the Cloudflare API, and piped straight into `wrangler secret put`; they are never printed. The script lists existing secrets first and stops with `unexpected output from wrangler secret list` if it can't read them, rather than risk overwriting `MASTER_KEY`, which would make every stored Actions secret unreadable. It never sets or removes `MASTER_KEY_PREV`: you set that yourself while [rotating MASTER_KEY](/docs/self-hosting/configuration#rotating-master_key).
189189
190190 ### Finishing Access on the first deploy
191191
@@ -279,7 +279,7 @@
279279 - Rebuilds the documentation from the Markdown files in your copy, so `/docs` matches the code you deploy.
280280 - Rolls out a new runner image. The first sandbox started on a machine after a deploy pulls the new image first, so it can take a few minutes to start; Diffinery waits up to 3 minutes for it. Running hosted jobs and imports keep their containers until they finish, and the next ones start on the new image. A sandbox up for more than an hour is replaced.
281281
282-Keep `MASTER_KEY` safe and never change it: it decrypts every stored Actions secret (repository, organization and environment secrets), unsubscribe links in emails already sent are signed with it, and so are the cache and artifact tokens of jobs that are running. If you delete `OIDC_PRIVATE_KEY`, the next deploy generates a new key with a new key ID; systems that trust your OIDC tokens fetch the new public key from `/.well-known/jwks`.
282+Keep `MASTER_KEY` safe and don't simply replace it: it decrypts every stored Actions secret (repository, organization and environment secrets), unsubscribe links in emails already sent are signed with it, and so are the cache and artifact tokens of jobs that are running. To change it, keep the old key as `MASTER_KEY_PREV` as described in [Rotating MASTER_KEY](/docs/self-hosting/configuration#rotating-master_key). If you delete `OIDC_PRIVATE_KEY`, the next deploy generates a new key with a new key ID; systems that trust your OIDC tokens fetch the new public key from `/.well-known/jwks`.
283283
284284 ## Local development
285285
modifiedapps/diffinery/src/agentic/keys.ts+2−2View file
@@ -99,7 +99,7 @@
9999 const k = key.trim();
100100 if (!PROVIDERS[provider].pattern.test(k)) throw new HttpError(422, `that doesn't look like an ${PROVIDERS[provider].name} API key`);
101101 const at = now();
102- const ciphertext = await encryptSecret(this.env.MASTER_KEY, scope(account.id, provider), k);
102+ const ciphertext = await encryptSecret(this.env, scope(account.id, provider), k);
103103 const hint = k.slice(-4);
104104 await this.env.DB.prepare(
105105 "INSERT INTO model_keys (account_id, provider, ciphertext, hint, opted_in_at, opted_in_by, created_at, updated_at) VALUES (?, ?, ?, ?, ?, ?, ?, ?) ON CONFLICT (account_id, provider) DO UPDATE SET ciphertext = excluded.ciphertext, hint = excluded.hint, opted_in_at = excluded.opted_in_at, opted_in_by = excluded.opted_in_by, updated_at = excluded.updated_at",
@@ -120,6 +120,6 @@
120120 /** The key itself: only for the model proxy, inside the request it forwards, while a task this account pays for runs. */
121121 async reveal(accountId: string, provider: Provider): Promise<string | null> {
122122 const row = await this.env.DB.prepare("SELECT ciphertext FROM model_keys WHERE account_id = ? AND provider = ?").bind(accountId, provider).first<{ ciphertext: string }>();
123- return row ? decryptSecret(this.env.MASTER_KEY, scope(accountId, provider), row.ciphertext) : null;
123+ return row ? decryptSecret(this.env, scope(accountId, provider), row.ciphertext) : null;
124124 }
125125 }
modifiedapps/diffinery/src/auth/invite-proof.ts+11−8View file
@@ -1,27 +1,30 @@
11 import { devAuth } from "./dev.js";
22 import type { Env } from "../env.js";
3-import { b64url, safeEqual } from "../lib/util.js";
3+import { hmacSha256, matchesAnyKey, signingSecrets } from "../lib/crypto.js";
4+import { b64url } from "../lib/util.js";
45
56 /**
67 * Proof that someone can read the invited address's email. The invite link shown to the
78 * person who sent it creates nothing on its own: only the copy emailed to the invited
89 * address carries this proof, so nobody can join as an address they don't own (and so
910 * collect its other invitations, or a bootstrap administrator's rights).
10- * The key is derived from MASTER_KEY, so emailed links stay valid across deploys.
11+ * The key is derived from MASTER_KEY, so emailed links stay valid across deploys, and across a
12+ * rotation of MASTER_KEY while the old one is kept as MASTER_KEY_PREV.
1113 */
12-async function key(env: Env): Promise<CryptoKey> {
14+function secrets(env: Env): string[] {
1315 // Local development has no MASTER_KEY unless one is passed; production always does.
14- const secret = env.MASTER_KEY ?? (devAuth(env) ? "diffinery-dev-invite-proof" : null);
15- if (!secret) throw new Error("MASTER_KEY is not set, so invitation emails can't be signed");
16- return crypto.subtle.importKey("raw", new TextEncoder().encode(`diffinery-invite-proof-v1:${secret}`), { name: "HMAC", hash: "SHA-256" }, false, ["sign"]);
16+ return signingSecrets(env, devAuth(env) ? "diffinery-dev-invite-proof" : null, "MASTER_KEY is not set, so invitation emails can't be signed");
1717 }
1818
19+const proofWith = async (secret: string, tokenHash: string): Promise<string> => b64url(await hmacSha256(`diffinery-invite-proof-v1:${secret}`, tokenHash));
20+
1921 /** The proof for the invite whose token hashes to `tokenHash`. */
2022 export async function inviteProof(env: Env, tokenHash: string): Promise<string> {
21- return b64url(new Uint8Array(await crypto.subtle.sign("HMAC", await key(env), new TextEncoder().encode(tokenHash))));
23+ return proofWith(secrets(env)[0]!, tokenHash);
2224 }
2325
26+/** Whether `proof` is the invite's proof, made with MASTER_KEY or, while it is being rotated, MASTER_KEY_PREV. */
2427 export async function validInviteProof(env: Env, tokenHash: string, proof: string | undefined | null): Promise<boolean> {
2528 if (!proof || proof.length > 100) return false;
26- return safeEqual(proof, await inviteProof(env, tokenHash));
29+ return matchesAnyKey(secrets(env), proof, (s) => proofWith(s, tokenHash));
2730 }
modifiedapps/diffinery/src/auth/signup.ts+16−7View file
@@ -1,7 +1,8 @@
11 import { loopback } from "./dev.js";
22 import type { Env } from "../env.js";
33 import type { Account } from "../lib/directory.js";
4-import { HttpError, b64url, newId, now, randomToken, safeEqual, sha256 } from "../lib/util.js";
4+import { hmacSha256, matchesAnyKey, signingSecrets } from "../lib/crypto.js";
5+import { HttpError, b64url, newId, now, randomToken, sha256 } from "../lib/util.js";
56
67 /**
78 * Open sign-up (SIGNUP=open). Someone asks for a code at /signup, proves they read mail at
@@ -72,14 +73,22 @@
7273 * The stored form of a code: an HMAC keyed from MASTER_KEY over the code's row id and the
7374 * code, so a copy of the database alone can't try the million codes.
7475 */
75-export async function codeHash(env: Pick<Env, "MASTER_KEY" | "PUBLIC_URL">, id: string, code: string): Promise<string> {
76+export async function codeHash(env: Pick<Env, "MASTER_KEY" | "MASTER_KEY_PREV" | "PUBLIC_URL">, id: string, code: string): Promise<string> {
77+ return codeHashWith(codeSecrets(env)[0]!, id, code);
78+}
79+
80+/** Whether `code` is the one stored as `hash`, keyed with MASTER_KEY or, while it is being rotated, MASTER_KEY_PREV. */
81+export async function codeMatches(env: Pick<Env, "MASTER_KEY" | "MASTER_KEY_PREV" | "PUBLIC_URL">, id: string, code: string, hash: string): Promise<boolean> {
82+ return matchesAnyKey(codeSecrets(env), hash, (s) => codeHashWith(s, id, code));
83+}
84+
85+function codeSecrets(env: Pick<Env, "MASTER_KEY" | "MASTER_KEY_PREV" | "PUBLIC_URL">): string[] {
7686 // Local development has no MASTER_KEY unless one is passed; production always does.
77- const secret = env.MASTER_KEY ?? (loopback(env.PUBLIC_URL) ? "diffinery-dev-signup" : null);
78- if (!secret) throw new Error("MASTER_KEY is not set, so sign-up codes can't be stored");
79- const key = await crypto.subtle.importKey("raw", new TextEncoder().encode(`diffinery-signup-code-v1:${secret}`), { name: "HMAC", hash: "SHA-256" }, false, ["sign"]);
80- return b64url(new Uint8Array(await crypto.subtle.sign("HMAC", key, new TextEncoder().encode(`${id}:${code}`))));
87+ return signingSecrets(env, loopback(env.PUBLIC_URL) ? "diffinery-dev-signup" : null, "MASTER_KEY is not set, so sign-up codes can't be stored");
8188 }
8289
90+const codeHashWith = async (secret: string, id: string, code: string): Promise<string> => b64url(await hmacSha256(`diffinery-signup-code-v1:${secret}`, `${id}:${code}`));
91+
8392 const minutes = (n: number): string => new Date(Date.now() + n * 60_000).toISOString();
8493
8594 export type CodeCheck = { ok: true } | { ok: false; left: number };
@@ -137,7 +146,7 @@
137146 .bind(email, t, MAX_CODE_ATTEMPTS)
138147 .first<{ id: string; code_hash: string; attempts: number }>();
139148 if (!row) return { ok: false, left: 0 };
140- if (safeEqual(row.code_hash, await codeHash(this.env, row.id, code))) {
149+ if (await codeMatches(this.env, row.id, code, row.code_hash)) {
141150 const r = await this.db.prepare("UPDATE signup_codes SET used_at = ? WHERE id = ? AND used_at IS NULL").bind(t, row.id).run();
142151 if ((r.meta.changes ?? 0) > 0) return { ok: true };
143152 }
modifiedapps/diffinery/src/env.ts+6−1View file
@@ -109,8 +109,13 @@
109109 */
110110 TURNSTILE_SITE_KEY?: string;
111111
112- /** Secret: base64 32 bytes; root key for envelope encryption of secrets. */
112+ /** Secret: base64 32 bytes; root key for envelope encryption of secrets and for the links and tokens Diffinery signs. */
113113 MASTER_KEY?: string;
114+ /**
115+ * Secret, set only while rotating MASTER_KEY: the key it replaced. Secrets sealed and links or tokens
116+ * signed with it keep working; everything new uses MASTER_KEY (lib/crypto.ts).
117+ */
118+ MASTER_KEY_PREV?: string;
114119 /** Secret: ES256 private JWK (JSON) used to sign OIDC tokens for jobs. */
115120 OIDC_PRIVATE_KEY?: string;
116121 /** Secret: signs bounce and complaint events posted to /-/email/events (mail/events.ts); unset, that endpoint is off. */
modifiedapps/diffinery/src/http/actions-runtime.ts+7−7View file
@@ -34,7 +34,7 @@
3434
3535 async function runtimeClaims(c: AppContext): Promise<RuntimeClaims> {
3636 const m = /^Bearer\s+(\S+)$/i.exec(c.req.header("authorization") ?? "");
37- const claims = m ? await verifyRuntimeToken(c.env.MASTER_KEY, m[1]!) : null;
37+ const claims = m ? await verifyRuntimeToken(c.env, m[1]!) : null;
3838 if (!claims) throw new HttpError(401, "invalid or expired runtime token");
3939 return claims;
4040 }
@@ -49,7 +49,7 @@
4949 return { status: 500, message: "internal error" };
5050 }
5151
52-const readUrl = (env: Env, b: BlobRef, fileName?: string) => signBlobUrl(env.MASTER_KEY, origin(env), b, "r", nowS() + READ_URL_TTL_S, fileName);
52+const readUrl = (env: Env, b: BlobRef, fileName?: string) => signBlobUrl(env, origin(env), b, "r", nowS() + READ_URL_TTL_S, fileName);
5353
5454 /**
5555 * A cache lookup whose object is really there. An entry whose object went missing
@@ -173,7 +173,7 @@
173173 twirp("CacheService", "CreateCacheEntry", async (c, claims, req) => {
174174 const reserved = await repoStub(c.env, claims.repo).rtCacheReserve(claims.job, str(req, "key", 512), str(req, "version", 128));
175175 if (!reserved) throw new HttpError(409, "a cache entry with this key and version already exists");
176- const url = await signBlobUrl(c.env.MASTER_KEY, origin(c.env), { repo: claims.repo, kind: "cache", id: reserved.id }, "w", claims.exp);
176+ const url = await signBlobUrl(c.env, origin(c.env), { repo: claims.repo, kind: "cache", id: reserved.id }, "w", claims.exp);
177177 return { ok: true, signed_upload_url: url };
178178 });
179179
@@ -207,7 +207,7 @@
207207 const mimeType = typeof mime === "string" && /^[\w.+-]{1,64}\/[\w.+-]{1,64}$/.test(mime) ? mime : ZIP;
208208 const expires = field(req, "expires_at");
209209 const { id } = await repoStub(c.env, claims.repo).rtArtifactCreate(claims.job, artifactName(req), mimeType, typeof expires === "string" ? expires : undefined);
210- const url = await signBlobUrl(c.env.MASTER_KEY, origin(c.env), { repo: claims.repo, kind: "artifact", id }, "w", claims.exp);
210+ const url = await signBlobUrl(c.env, origin(c.env), { repo: claims.repo, kind: "artifact", id }, "w", claims.exp);
211211 return { ok: true, signed_upload_url: url };
212212 });
213213
@@ -426,7 +426,7 @@
426426 const b = blobRef(c);
427427 if (!b) return azureError(404, "ResourceNotFound", "The specified resource does not exist.");
428428 const q = new URL(c.req.url).searchParams;
429- if (!(await verifyBlobUrl(c.env.MASTER_KEY, b, "w", q))) return azureError(403, "AuthenticationFailed", "The upload URL is invalid or has expired.");
429+ if (!(await verifyBlobUrl(c.env, b, "w", q))) return azureError(403, "AuthenticationFailed", "The upload URL is invalid or has expired.");
430430 try {
431431 const comp = q.get("comp");
432432 if (comp === "block") {
@@ -471,7 +471,7 @@
471471 const serveBlob = async (c: AppContext) => {
472472 const b = blobRef(c);
473473 if (!b) return azureError(404, "ResourceNotFound", "The specified resource does not exist.");
474- if (!(await verifyBlobUrl(c.env.MASTER_KEY, b, "r", new URL(c.req.url).searchParams))) return azureError(403, "AuthenticationFailed", "The download URL is invalid or has expired.");
474+ if (!(await verifyBlobUrl(c.env, b, "r", new URL(c.req.url).searchParams))) return azureError(403, "AuthenticationFailed", "The download URL is invalid or has expired.");
475475 const stub = repoStub(c.env, b.repo);
476476 const headers: Record<string, string> = { "accept-ranges": "bytes", "x-ms-blob-type": "BlockBlob", "content-type": "application/octet-stream" };
477477 let etag: string | null;
@@ -576,7 +576,7 @@
576576 if (!a) throw new HttpError(404, "Not Found");
577577 // As on GitHub: an expired artifact is still listed, but its content is gone.
578578 if (a.expired) throw new HttpError(410, "Artifact has expired");
579- return c.redirect(await signBlobUrl(c.env.MASTER_KEY, origin(c.env), { repo: repo.id, kind: "artifact", id: a.id }, "r", nowS() + 600, artifactFileName(a)), 302);
579+ return c.redirect(await signBlobUrl(c.env, origin(c.env), { repo: repo.id, kind: "artifact", id: a.id }, "r", nowS() + 600, artifactFileName(a)), 302);
580580 });
581581
582582 actionsRest.delete("/repos/:owner/:repo/actions/artifacts/:id", async (c) => {
modifiedapps/diffinery/src/http/api.ts+1−1View file
@@ -1288,7 +1288,7 @@
12881288 // A new secret reaches every repository unless the request says otherwise; an update keeps its selection.
12891289 const visibility = (b.visibility as OrgSecretRow["visibility"] | undefined) ?? existing?.visibility ?? "all";
12901290 const repos = b.selected_repository_ids !== undefined ? await orgReposById(c, org, b.selected_repository_ids) : null;
1291- await c.env.DB.prepare("INSERT OR REPLACE INTO org_secrets (org_id, name, ciphertext, updated_at, visibility) VALUES (?, ?, ?, ?, ?)").bind(org.id, name, await encryptSecret(c.env.MASTER_KEY, `org:${org.id}`, b.value), new Date().toISOString(), visibility).run();
1291+ await c.env.DB.prepare("INSERT OR REPLACE INTO org_secrets (org_id, name, ciphertext, updated_at, visibility) VALUES (?, ?, ?, ?, ?)").bind(org.id, name, await encryptSecret(c.env, `org:${org.id}`, b.value), new Date().toISOString(), visibility).run();
12921292 if (repos) await setSecretRepos(c, org, name, repos);
12931293 else if (visibility !== "selected") await c.env.DB.prepare("DELETE FROM org_secret_repos WHERE org_id = ? AND name = ?").bind(org.id, name).run();
12941294 c.executionCtx.waitUntil(c.get("dir").audit({ actorId: a.id, actorLogin: a.login, action: "org.secret.set", target: org.login, detail: { name, visibility, ...(repos ? { repositories: repos.map((r) => r.name) } : {}), via: "api" } }));
modifiedapps/diffinery/src/imports/service.ts+36−26View file
@@ -2,7 +2,7 @@
22 import { NonRetryableError } from "cloudflare:workflows";
33 import type { ImportGitResult, ImportProgress, ImportSecretFinding, ImportSpec } from "@diffinery/shared";
44 import type { Env } from "../env.js";
5-import { decryptSecret, encryptSecret } from "../lib/crypto.js";
5+import { decryptSecret, encryptSecret, hmacSha256, matchesAnyKey, resealSecret, signingSecrets } from "../lib/crypto.js";
66 import { Directory, type Account, type Repo, type TokenScope } from "../lib/directory.js";
77 import { atLeast } from "../repo/types.js";
88 import { HttpError, newId, now } from "../lib/util.js";
@@ -174,6 +174,8 @@
174174 const DEFAULT_MAX_REPO_KB = 10 * 1024 * 1024;
175175
176176 const fatal = (message: string) => new NonRetryableError(message);
177+/** The first 16 bytes of a MAC in hex (gist nonces and OAuth state). */
178+const hex16 = (mac: Uint8Array): string => [...mac.slice(0, 16)].map((b) => b.toString(16).padStart(2, "0")).join("");
177179 const errMsg = (e: unknown) => (e instanceof Error ? e.message.replace(/^\d{3}: /, "") : String(e));
178180
179181 /** A valid Actions variable name: GitHub's rules (letters, digits, underscore, not starting with a digit) and never the reserved GITHUB_ prefix. */
@@ -423,7 +425,7 @@
423425 };
424426 const id = newId();
425427 const t = now();
426- const ciphertext = req.token ? await encryptSecret(this.env.MASTER_KEY, `import:${id}`, req.token) : null;
428+ const ciphertext = req.token ? await encryptSecret(this.env, `import:${id}`, req.token) : null;
427429 const insert = (r: Partial<ImportRow> & { id: string; kind: ImportRow["kind"]; status: ImportStatus; source: string }) =>
428430 this.db
429431 .prepare(
@@ -494,7 +496,7 @@
494496 if (chosen.length > MAX_BULK_REPOS) throw new HttpError(422, `at most ${MAX_BULK_REPOS} repositories per import`);
495497 const owner = (await this.dir.account(row.target_owner_id))!;
496498 if (!(await this.canImportInto(actor, owner))) throw new HttpError(403, `you can no longer import into ${owner.login}`);
497- const token = row.token_ciphertext ? await decryptSecret(this.env.MASTER_KEY, `import:${id}`, row.token_ciphertext) : null;
499+ const token = row.token_ciphertext ? await decryptSecret(this.env, `import:${id}`, row.token_ciphertext) : null;
498500 const { candidates: _c, ...rest } = req;
499501 await this.db.prepare("UPDATE imports SET status = 'queued', request = ?, updated_at = ? WHERE id = ?").bind(JSON.stringify({ ...rest, repositories: chosen }), now(), id).run();
500502 await this.createChildren((await this.row(id))!, owner, chosen, token);
@@ -526,7 +528,7 @@
526528 name,
527529 parent.request,
528530 clash ? `${owner.login}/${name} already exists` : null,
529- token && !clash ? await encryptSecret(this.env.MASTER_KEY, `import:${id}`, token) : null,
531+ token && !clash ? await encryptSecret(this.env, `import:${id}`, token) : null,
530532 t,
531533 t,
532534 clash ? t : null,
@@ -614,7 +616,8 @@
614616 const attempt = row.attempt + 1;
615617 const t = now();
616618 const reset = async (r: ImportRow, n: number) => {
617- const ciphertext = token ? await encryptSecret(this.env.MASTER_KEY, `import:${r.id}`, token) : r.token_ciphertext;
619+ // A kept token sealed with MASTER_KEY_PREV (or as v1) is sealed again with MASTER_KEY, as the row is written anyway.
620+ const ciphertext = token ? await encryptSecret(this.env, `import:${r.id}`, token) : r.token_ciphertext ? await resealSecret(this.env, `import:${r.id}`, r.token_ciphertext) : r.token_ciphertext;
618621 await this.db
619622 .prepare("UPDATE imports SET status = 'queued', attempt = ?, error = NULL, steps = '{}', warnings = '[]', token_ciphertext = ?, started_at = NULL, finished_at = NULL, updated_at = ? WHERE id = ?")
620623 .bind(n, ciphertext, t, r.id)
@@ -700,7 +703,7 @@
700703 private async context(id: string): Promise<Ctx> {
701704 const row = await this.mustRow(id);
702705 if (row.status !== "running") throw fatal(`import is ${row.status}`);
703- const token = row.token_ciphertext ? await decryptSecret(this.env.MASTER_KEY, `import:${id}`, row.token_ciphertext) : null;
706+ const token = row.token_ciphertext ? await decryptSecret(this.env, `import:${id}`, row.token_ciphertext) : null;
704707 // Links are keyed by the full normalised API base, so github.com and an Enterprise host never mix.
705708 const apiBase = normalizeApiUrl(row.api_url, this.dev);
706709 const links = await this.db
@@ -1397,7 +1400,7 @@
13971400 return {
13981401 importId: id,
13991402 source: { fullName: row.source, cloneUrl: cloneUrl.toString(), defaultBranch: summary.source.defaultBranch },
1400- githubToken: row.token_ciphertext ? await decryptSecret(this.env.MASTER_KEY, `import:${id}`, row.token_ciphertext) : null,
1403+ githubToken: row.token_ciphertext ? await decryptSecret(this.env, `import:${id}`, row.token_ciphertext) : null,
14011404 target: { fullName: `${repo.owner_login}/${repo.name}`, pushUrl: `${this.env.PUBLIC_URL}/${repo.owner_login}/${repo.name}.git` },
14021405 pushToken,
14031406 lfs: req.options.lfs,
@@ -1480,16 +1483,23 @@
14801483 async linkChallenge(account: Account, apiUrlRaw: string | null): Promise<{ nonce: string; apiBase: string; host: string; gistsUrl: string }> {
14811484 if (account.kind !== "user") throw new HttpError(403, "only people can link GitHub accounts");
14821485 const apiBase = normalizeApiUrl(apiUrlRaw, this.dev);
1483- const nonce = await this.linkNonce(account.id, apiBase);
1484- return { nonce, apiBase, host: webHostOf(apiBase), gistsUrl: `${apiBase}/gists` };
1486+ const [nonce] = await this.linkNonces(account.id, apiBase);
1487+ return { nonce: nonce!, apiBase, host: webHostOf(apiBase), gistsUrl: `${apiBase}/gists` };
14851488 }
14861489
1487- private async linkNonce(accountId: string, apiBase: string): Promise<string> {
1488- const key = await crypto.subtle.importKey("raw", new TextEncoder().encode(this.env.MASTER_KEY ?? "dev"), { name: "HMAC", hash: "SHA-256" }, false, ["sign"]);
1489- const mac = await crypto.subtle.sign("HMAC", key, new TextEncoder().encode(`github-link:${accountId}:${apiBase}`));
1490- return `diffinery-link-${[...new Uint8Array(mac)].slice(0, 16).map((b) => b.toString(16).padStart(2, "0")).join("")}`;
1490+ /**
1491+ * The nonce under MASTER_KEY first, the one to show; then, while MASTER_KEY is being rotated, the one
1492+ * under MASTER_KEY_PREV, which a gist made before the rotation carries and which still proves the link.
1493+ */
1494+ private async linkNonces(accountId: string, apiBase: string): Promise<string[]> {
1495+ return Promise.all(this.linkSecrets().map(async (k) => `diffinery-link-${hex16(await hmacSha256(k, `github-link:${accountId}:${apiBase}`))}`));
14911496 }
14921497
1498+ /** The keys of gist nonces and OAuth state: the master keys, or "dev" when none is set. */
1499+ private linkSecrets(): string[] {
1500+ return signingSecrets(this.env, "dev", "");
1501+ }
1502+
14931503 /**
14941504 * Link a GitHub account by gist proof (no token): the person's nonce must appear in one of their
14951505 * PUBLIC gists, read unauthenticated. The numeric id comes from the public user profile, so a
@@ -1499,7 +1509,7 @@
14991509 if (account.kind !== "user") throw new HttpError(403, "only people can link GitHub accounts");
15001510 if (!/^[A-Za-z0-9](?:[A-Za-z0-9-]{0,38})$/.test(login)) throw new HttpError(422, "that is not a valid GitHub login");
15011511 const apiBase = normalizeApiUrl(apiUrlRaw, this.dev);
1502- const nonce = await this.linkNonce(account.id, apiBase);
1512+ const nonces = await this.linkNonces(account.id, apiBase);
15031513 const gh = new GitHubClient(apiBase, null); // unauthenticated: public data only
15041514 let gists: Array<{ id: string; description?: string | null; public?: boolean }>;
15051515 let user: { login: string; id: number };
@@ -1512,26 +1522,27 @@
15121522 throw new HttpError(502, `could not reach GitHub: ${errMsg(e)}`);
15131523 }
15141524 if (!user?.login || user.id === undefined) throw new HttpError(502, "GitHub did not return a user");
1515- const proven = await this.gistHasNonce(gh, gists, nonce);
1516- if (!proven) throw new HttpError(422, `create a PUBLIC gist whose description or a file contains ${nonce}, then try again`);
1525+ const proven = await this.gistHasNonce(gh, gists, nonces);
1526+ if (!proven) throw new HttpError(422, `create a PUBLIC gist whose description or a file contains ${nonces[0]}, then try again`);
15171527 return this.saveLink(account, apiBase, String(user.id), user.login);
15181528 }
15191529
15201530 /**
1521- * Whether one of the user's public gists contains the nonce. The gist bodies are read through the
1531+ * Whether one of the user's public gists contains one of the nonces. The gist bodies are read through the
15221532 * same API host (GET /gists/{id}), never a cross-host raw URL, so no credential or request leaves
15231533 * the allowlisted host.
15241534 */
1525- private async gistHasNonce(gh: GitHubClient, gists: Array<{ id: string; description?: string | null; public?: boolean }>, nonce: string): Promise<boolean> {
1535+ private async gistHasNonce(gh: GitHubClient, gists: Array<{ id: string; description?: string | null; public?: boolean }>, nonces: string[]): Promise<boolean> {
1536+ const has = (text: string) => nonces.some((n) => text.includes(n));
15261537 let fetched = 0;
15271538 for (const g of gists.slice(0, 100)) {
15281539 if (g.public === false || !g.id) continue;
1529- if ((g.description ?? "").includes(nonce)) return true;
1540+ if (has(g.description ?? "")) return true;
15301541 if (fetched >= 20) continue; // bound how many gist bodies we read
15311542 fetched++;
15321543 try {
15331544 const full = await gh.get<{ files?: Record<string, { content?: string | null }> }>(`gists/${encodeURIComponent(g.id)}`);
1534- for (const f of Object.values(full.files ?? {})) if ((f.content ?? "").includes(nonce)) return true;
1545+ for (const f of Object.values(full.files ?? {})) if (has(f.content ?? "")) return true;
15351546 } catch {
15361547 // skip an unreadable gist
15371548 }
@@ -1561,7 +1572,7 @@
15611572 /** The GitHub authorize URL with a state bound (HMAC) to this account, so a callback cannot be replayed for another. */
15621573 async oauthStart(account: Account): Promise<string> {
15631574 if (!this.oauthEnabled) throw new HttpError(404, "GitHub OAuth is not configured");
1564- const state = await this.oauthState(account.id);
1575+ const state = await this.oauthState(this.linkSecrets()[0]!, account.id);
15651576 const u = new URL("https://github.com/login/oauth/authorize");
15661577 u.searchParams.set("client_id", this.env.GITHUB_OAUTH_CLIENT_ID!);
15671578 u.searchParams.set("redirect_uri", `${this.env.PUBLIC_URL}/settings/linked/github/callback`);
@@ -1571,15 +1582,14 @@
15711582 return u.toString();
15721583 }
15731584
1574- private async oauthState(accountId: string): Promise<string> {
1575- const key = await crypto.subtle.importKey("raw", new TextEncoder().encode(this.env.MASTER_KEY ?? "dev"), { name: "HMAC", hash: "SHA-256" }, false, ["sign"]);
1576- const mac = await crypto.subtle.sign("HMAC", key, new TextEncoder().encode(`github-oauth:${accountId}`));
1577- return `${accountId}.${[...new Uint8Array(mac)].slice(0, 16).map((b) => b.toString(16).padStart(2, "0")).join("")}`;
1585+ private async oauthState(secret: string, accountId: string): Promise<string> {
1586+ return `${accountId}.${hex16(await hmacSha256(secret, `github-oauth:${accountId}`))}`;
15781587 }
15791588
15801589 async oauthCallback(account: Account, code: string, state: string): Promise<{ id: string; host: string; login: string }> {
15811590 if (!this.oauthEnabled) throw new HttpError(404, "GitHub OAuth is not configured");
1582- if (!code || !state || state !== (await this.oauthState(account.id))) throw new HttpError(422, "invalid or expired OAuth state");
1591+ // A flow started just before MASTER_KEY was rotated carries a state made with MASTER_KEY_PREV.
1592+ if (!code || !state || !(await matchesAnyKey(this.linkSecrets(), state, (k) => this.oauthState(k, account.id)))) throw new HttpError(422, "invalid or expired OAuth state");
15831593 const tokenRes = await fetch("https://github.com/login/oauth/access_token", {
15841594 method: "POST",
15851595 headers: { accept: "application/json", "content-type": "application/json" },
modifiedapps/diffinery/src/lib/crypto.ts+144−10View file
@@ -1,9 +1,82 @@
1-import { b64, fromB64 } from "./util.js";
1+import { b64, fromB64, safeEqual } from "./util.js";
22
33 /**
4+ * Everything Diffinery encrypts or signs with its own key starts from MASTER_KEY. To rotate it, an
5+ * operator moves the old value to MASTER_KEY_PREV and sets a new MASTER_KEY: from then on, new
6+ * ciphertexts and signatures use MASTER_KEY, and what was made with the old key still decrypts or
7+ * verifies until MASTER_KEY_PREV is removed (docs/self-hosting/configuration.md, "Rotating MASTER_KEY").
8+ */
9+export interface MasterKeyEnv {
10+ /** Secret: base64 32 bytes; the current root key. */
11+ MASTER_KEY?: string;
12+ /** Secret: the root key MASTER_KEY replaced, still accepted while a rotation is under way. */
13+ MASTER_KEY_PREV?: string;
14+}
15+
16+/**
17+ * The configured master keys, the current one first and then the previous one while a rotation is
18+ * under way. Empty without MASTER_KEY: MASTER_KEY_PREV alone never makes a key current.
19+ */
20+export function masterKeys(env: MasterKeyEnv): string[] {
21+ const current = env.MASTER_KEY;
22+ if (!current) return [];
23+ const prev = env.MASTER_KEY_PREV;
24+ return prev && prev !== current ? [current, prev] : [current];
25+}
26+
27+/**
28+ * The secrets a signer derives its key from: the master keys, or `devFallback` when no MASTER_KEY is
29+ * set (local development only; callers pass null outside it, and then this throws `missing`).
30+ */
31+export function signingSecrets(env: MasterKeyEnv, devFallback: string | null, missing: string): string[] {
32+ const keys = masterKeys(env);
33+ if (keys.length) return keys;
34+ if (devFallback !== null) return [devFallback];
35+ throw new Error(missing);
36+}
37+
38+/**
39+ * Whether `tag` is what `mac` computes under any of `secrets`. New tags are always made with the
40+ * first (current) secret; this is the matching check that also takes a tag made with the previous
41+ * one. An error computing the current tag propagates as before rotation existed; one under the
42+ * previous secret (say a MASTER_KEY_PREV that isn't valid base64) only means "no match".
43+ */
44+export async function matchesAnyKey(secrets: readonly string[], tag: string, mac: (secret: string) => Promise<string>): Promise<boolean> {
45+ for (const [i, s] of secrets.entries()) {
46+ let expected: string;
47+ try {
48+ expected = await mac(s);
49+ } catch (e) {
50+ if (i === 0) throw e;
51+ continue;
52+ }
53+ if (safeEqual(expected, tag)) return true;
54+ }
55+ return false;
56+}
57+
58+/** HMAC-SHA256 of `data` under the UTF-8 bytes of `secret`. */
59+export async function hmacSha256(secret: string, data: string): Promise<Uint8Array> {
60+ const key = await crypto.subtle.importKey("raw", new TextEncoder().encode(secret), { name: "HMAC", hash: "SHA-256" }, false, ["sign"]);
61+ return new Uint8Array(await crypto.subtle.sign("HMAC", key, new TextEncoder().encode(data)));
62+}
63+
64+// ---------------------------------------------------------------- envelope encryption
65+
66+/**
467 * Envelope encryption for secrets. Each scope (repo or org) gets its own AES-GCM
568 * key derived from MASTER_KEY with HKDF, so a leaked ciphertext from one repo
669 * cannot be replayed into another.
70+ *
71+ * Envelopes:
72+ * - `v2.<kid>.<iv>.<ct>`, written now. `kid` names the master key that sealed it: the first 8 hex
73+ * characters of SHA-256 over the key's raw (base64-decoded) bytes, which an operator can compute with
74+ * `printf %s "$KEY" | base64 -d | sha256sum | cut -c1-8`. It identifies the key, it isn't secret,
75+ * and it says nothing useful about a 256-bit random key.
76+ * - `v1.<iv>.<ct>`, written before keys could be rotated; it says nothing about its key, so both
77+ * configured keys are tried.
78+ * The AES-GCM key, IV size and additional data (the scope) are the same in both, so a v1 envelope's
79+ * ciphertext is exactly what v2 would carry.
780 */
881 async function scopeKey(masterKeyB64: string, scope: string): Promise<CryptoKey> {
982 const raw = await crypto.subtle.importKey("raw", fromB64(masterKeyB64) as BufferSource, "HKDF", false, ["deriveKey"]);
@@ -16,19 +89,80 @@
1689 );
1790 }
1891
19-export async function encryptSecret(masterKey: string | undefined, scope: string, plaintext: string): Promise<string> {
92+const kids = new Map<string, Promise<string>>();
93+
94+/** The key id that v2 envelopes sealed with this master key carry. */
95+export function keyId(masterKeyB64: string): Promise<string> {
96+ let id = kids.get(masterKeyB64);
97+ if (!id) {
98+ id = (async () => {
99+ const d = new Uint8Array(await crypto.subtle.digest("SHA-256", fromB64(masterKeyB64) as BufferSource));
100+ return [...d.slice(0, 4)].map((x) => x.toString(16).padStart(2, "0")).join("");
101+ })();
102+ // A key that isn't base64 rejects; don't keep that around.
103+ id.catch(() => kids.delete(masterKeyB64));
104+ kids.set(masterKeyB64, id);
105+ }
106+ return id;
107+}
108+
109+export async function encryptSecret(env: MasterKeyEnv, scope: string, plaintext: string): Promise<string> {
110+ const [masterKey] = masterKeys(env);
20111 if (!masterKey) throw new Error("MASTER_KEY is not configured");
21112 const iv = crypto.getRandomValues(new Uint8Array(12));
22113 const key = await scopeKey(masterKey, scope);
23114 const ct = new Uint8Array(await crypto.subtle.encrypt({ name: "AES-GCM", iv, additionalData: new TextEncoder().encode(scope) }, key, new TextEncoder().encode(plaintext)));
24- return `v1.${b64(iv)}.${b64(ct)}`;
115+ return `v2.${await keyId(masterKey)}.${b64(iv)}.${b64(ct)}`;
25116 }
26117
27-export async function decryptSecret(masterKey: string | undefined, scope: string, envelope: string): Promise<string> {
28- if (!masterKey) throw new Error("MASTER_KEY is not configured");
29- const [v, ivB, ctB] = envelope.split(".");
30- if (v !== "v1" || !ivB || !ctB) throw new Error("bad secret envelope");
31- const key = await scopeKey(masterKey, scope);
32- const pt = await crypto.subtle.decrypt({ name: "AES-GCM", iv: fromB64(ivB) as BufferSource, additionalData: new TextEncoder().encode(scope) }, key, fromB64(ctB) as BufferSource);
33- return new TextDecoder().decode(pt);
118+export interface OpenedSecret {
119+ plaintext: string;
120+ /** Sealed with the current MASTER_KEY in the current format; false means it is worth sealing again. */
121+ current: boolean;
34122 }
123+
124+/** Decrypts an envelope with whichever configured master key sealed it, and says whether it is current. */
125+export async function openSecret(env: MasterKeyEnv, scope: string, envelope: string): Promise<OpenedSecret> {
126+ const keys = masterKeys(env);
127+ if (!keys.length) throw new Error("MASTER_KEY is not configured");
128+ const parts = envelope.split(".");
129+ const v1 = parts[0] === "v1" && parts.length === 3;
130+ const v2 = parts[0] === "v2" && parts.length === 4 && /^[0-9a-f]{8}$/.test(parts[1]!);
131+ const kid = v2 ? parts[1]! : null;
132+ const [ivB, ctB] = v2 ? parts.slice(2) : parts.slice(1);
133+ if ((!v1 && !v2) || !ivB || !ctB) throw new Error("bad secret envelope");
134+ // The key the kid names first; the others after it, in case two keys ever share a kid.
135+ const ids = await Promise.all(keys.map((k) => keyId(k).catch(() => null)));
136+ const order = kid === null ? keys.map((_, i) => i) : [...keys.keys()].sort((a, b) => Number(ids[b] === kid) - Number(ids[a] === kid));
137+ const iv = fromB64(ivB);
138+ const ct = fromB64(ctB);
139+ let failure: unknown = null;
140+ for (const i of order) {
141+ try {
142+ const key = await scopeKey(keys[i]!, scope);
143+ const pt = await crypto.subtle.decrypt({ name: "AES-GCM", iv: iv as BufferSource, additionalData: new TextEncoder().encode(scope) }, key, ct as BufferSource);
144+ return { plaintext: new TextDecoder().decode(pt), current: i === 0 && kid !== null && kid === ids[0] };
145+ } catch (e) {
146+ failure ??= e;
147+ }
148+ }
149+ throw new Error(`the secret doesn't decrypt with ${keys.length > 1 ? "MASTER_KEY or MASTER_KEY_PREV" : "MASTER_KEY"}`, { cause: failure });
150+}
151+
152+export async function decryptSecret(env: MasterKeyEnv, scope: string, envelope: string): Promise<string> {
153+ return (await openSecret(env, scope, envelope)).plaintext;
154+}
155+
156+/**
157+ * The envelope to store in place of `envelope` when a row is written anyway: the same secret sealed
158+ * with the current MASTER_KEY when it was sealed with the previous key or as v1, otherwise `envelope`
159+ * itself. One that doesn't decrypt is kept as it is, so writing a row never loses what it held.
160+ */
161+export async function resealSecret(env: MasterKeyEnv, scope: string, envelope: string): Promise<string> {
162+ try {
163+ const s = await openSecret(env, scope, envelope);
164+ return s.current ? envelope : await encryptSecret(env, scope, s.plaintext);
165+ } catch {
166+ return envelope;
167+ }
168+}
modifiedapps/diffinery/src/lib/runtime-auth.ts+21−14View file
@@ -1,4 +1,5 @@
1-import { b64url, fromB64, fromB64url, safeEqual } from "./util.js";
1+import { masterKeys, matchesAnyKey, type MasterKeyEnv } from "./crypto.js";
2+import { b64url, fromB64, fromB64url } from "./util.js";
23
34 /**
45 * Credentials for GitHub's Actions runtime services (cache, artifacts) as the
@@ -8,6 +9,8 @@
89 * backend ids from its `scp` claim, and buildkit's gha cache reads `ac`, `exp`
910 * and `nbf`. We sign it with HMAC-SHA256 under a key derived from MASTER_KEY,
1011 * scoped to one job of one repository, and it expires with the job.
12+ * While MASTER_KEY is being rotated, tokens and URLs signed under
13+ * MASTER_KEY_PREV still verify; new ones are signed under MASTER_KEY.
1114 * - Blob URLs (where the toolkit PUTs and GETs archives with the Azure SDK or
1215 * plain HTTP) are signed the same way, Azure SAS style: `se` (expiry) and
1316 * `sp` (r or w) plus `sig`.
@@ -24,8 +27,7 @@
2427
2528 const keys = new Map<string, Promise<CryptoKey>>();
2629
27-function runtimeKey(masterKey: string | undefined): Promise<CryptoKey> {
28- if (!masterKey) throw new Error("MASTER_KEY is not configured");
30+function runtimeKey(masterKey: string): Promise<CryptoKey> {
2931 let k = keys.get(masterKey);
3032 if (!k) {
3133 k = (async () => {
@@ -43,7 +45,14 @@
4345 return k;
4446 }
4547
46-async function hmac(masterKey: string | undefined, data: string): Promise<string> {
48+/** The configured master keys, current first; throws without MASTER_KEY. */
49+function keysOf(env: MasterKeyEnv): string[] {
50+ const all = masterKeys(env);
51+ if (!all.length) throw new Error("MASTER_KEY is not configured");
52+ return all;
53+}
54+
55+async function hmac(masterKey: string, data: string): Promise<string> {
4756 const sig = await crypto.subtle.sign("HMAC", await runtimeKey(masterKey), new TextEncoder().encode(data));
4857 return b64url(new Uint8Array(sig));
4958 }
@@ -54,7 +63,7 @@
5463 * Mints the job's ACTIONS_RUNTIME_TOKEN. `cacheScopes` are the refs whose caches
5564 * the job may read (the first one is also writable), in GitHub's `ac` format.
5665 */
57-export async function signRuntimeToken(masterKey: string | undefined, issuer: string, c: RuntimeClaims, cacheScopes: string[]): Promise<string> {
66+export async function signRuntimeToken(env: MasterKeyEnv, issuer: string, c: RuntimeClaims, cacheScopes: string[]): Promise<string> {
5867 const nowS = Math.floor(Date.now() / 1000);
5968 const ac = cacheScopes.map((s, i) => ({ Scope: s, Permission: i === 0 ? 3 : 1 }));
6069 const payload = {
@@ -71,20 +80,18 @@
7180 };
7281 const head = enc({ alg: "HS256", typ: "JWT" });
7382 const body = enc(payload);
74- return `${head}.${body}.${await hmac(masterKey, `${head}.${body}`)}`;
83+ return `${head}.${body}.${await hmac(keysOf(env)[0]!, `${head}.${body}`)}`;
7584 }
7685
7786 /** Verifies a runtime token; returns its claims, or null when it is invalid or expired. */
78-export async function verifyRuntimeToken(masterKey: string | undefined, token: string): Promise<RuntimeClaims | null> {
87+export async function verifyRuntimeToken(env: MasterKeyEnv, token: string): Promise<RuntimeClaims | null> {
7988 const [h, p, s] = token.split(".");
8089 if (!h || !p || !s) return null;
81- let expected: string;
8290 try {
83- expected = await hmac(masterKey, `${h}.${p}`);
91+ if (!(await matchesAnyKey(keysOf(env), s, (k) => hmac(k, `${h}.${p}`)))) return null;
8492 } catch {
8593 return null;
8694 }
87- if (!safeEqual(expected, s)) return null;
8895 try {
8996 const head = JSON.parse(new TextDecoder().decode(fromB64url(h))) as { alg?: string };
9097 if (head.alg !== "HS256") return null;
@@ -108,20 +115,20 @@
108115 const blobPayload = (b: BlobRef, perm: "r" | "w", exp: number) => `blob|${b.repo}|${b.kind}|${b.id}|${perm}|${exp}`;
109116
110117 /** A signed URL for reading (`r`) or writing (`w`) a cache entry's or artifact's content. */
111-export async function signBlobUrl(masterKey: string | undefined, origin: string, b: BlobRef, perm: "r" | "w", exp: number, fileName?: string): Promise<string> {
112- const sig = await hmac(masterKey, blobPayload(b, perm, exp));
118+export async function signBlobUrl(env: MasterKeyEnv, origin: string, b: BlobRef, perm: "r" | "w", exp: number, fileName?: string): Promise<string> {
119+ const sig = await hmac(keysOf(env)[0]!, blobPayload(b, perm, exp));
113120 // A trailing file name lets clients that look at the URL path (e.g. `.zip`) see the type.
114121 const tail = fileName ? `/${encodeURIComponent(fileName)}` : "";
115122 return `${origin}/_runtime/blob/${b.repo}/${b.kind}/${b.id}${tail}?sv=2025-01-05&se=${exp}&sp=${perm}&sig=${sig}`;
116123 }
117124
118125 /** Checks a blob URL's signature and expiry for the wanted permission. */
119-export async function verifyBlobUrl(masterKey: string | undefined, b: BlobRef, perm: "r" | "w", q: URLSearchParams): Promise<boolean> {
126+export async function verifyBlobUrl(env: MasterKeyEnv, b: BlobRef, perm: "r" | "w", q: URLSearchParams): Promise<boolean> {
120127 const exp = Number(q.get("se"));
121128 const sig = q.get("sig") ?? "";
122129 if (q.get("sp") !== perm || !Number.isInteger(exp) || exp < Math.floor(Date.now() / 1000)) return false;
123130 try {
124- return safeEqual(await hmac(masterKey, blobPayload(b, perm, exp)), sig);
131+ return await matchesAnyKey(keysOf(env), sig, (k) => hmac(k, blobPayload(b, perm, exp)));
125132 } catch {
126133 return false;
127134 }
modifiedapps/diffinery/src/mail/suppression.ts+6−9View file
@@ -1,6 +1,7 @@
11 import { devAuth } from "../auth/dev.js";
22 import type { Env } from "../env.js";
33 import { b64url, fromB64url, hmacSha256Hex, now, safeEqual } from "../lib/util.js";
4+import { hmacSha256, matchesAnyKey, signingSecrets } from "../lib/crypto.js";
45
56 /**
67 * The suppression list: addresses Diffinery sends no email to at all, because mail to them
@@ -180,28 +181,24 @@
180181 expiresAt: number;
181182 }
182183
183-async function verifyKey(env: Pick<Env, "MASTER_KEY" | "DEV_AUTH" | "ACCESS_AUD" | "PUBLIC_URL">): Promise<CryptoKey> {
184+function verifySecrets(env: Pick<Env, "MASTER_KEY" | "MASTER_KEY_PREV" | "DEV_AUTH" | "ACCESS_AUD" | "PUBLIC_URL">): string[] {
184185 // Local development has no MASTER_KEY unless one is passed; production always does.
185- const secret = env.MASTER_KEY ?? (devAuth(env) ? "diffinery-dev-verify-email" : null);
186- if (!secret) throw new Error("MASTER_KEY is not set, so verification links can't be signed");
187- return crypto.subtle.importKey("raw", new TextEncoder().encode(`diffinery-verify-email-v1:${secret}`), { name: "HMAC", hash: "SHA-256" }, false, ["sign"]);
186+ return signingSecrets(env, devAuth(env) ? "diffinery-dev-verify-email" : null, "MASTER_KEY is not set, so verification links can't be signed");
188187 }
189188
190-async function signVerify(env: Env, payload: string): Promise<string> {
191- return b64url(new Uint8Array(await crypto.subtle.sign("HMAC", await verifyKey(env), new TextEncoder().encode(payload))));
192-}
189+const signVerify = async (secret: string, payload: string): Promise<string> => b64url(await hmacSha256(`diffinery-verify-email-v1:${secret}`, payload));
193190
194191 /** A signed link token: this account may lift the suppression on this address until it expires. */
195192 export async function verifyToken(env: Env, t: VerifyToken): Promise<string> {
196193 const payload = b64url(new TextEncoder().encode(JSON.stringify({ a: t.accountId, e: normalAddress(t.email), x: t.expiresAt })));
197- return `${payload}.${await signVerify(env, payload)}`;
194+ return `${payload}.${await signVerify(verifySecrets(env)[0]!, payload)}`;
198195 }
199196
200197 /** The token's contents when its signature is good and it hasn't expired; null otherwise. */
201198 export async function readVerifyToken(env: Env, token: string, nowMs = Date.now()): Promise<VerifyToken | null> {
202199 const [payload, sig] = token.split(".");
203200 if (!payload || !sig || token.length > 1000) return null;
204- if (!safeEqual(sig, await signVerify(env, payload))) return null;
201+ if (!(await matchesAnyKey(verifySecrets(env), sig, (s) => signVerify(s, payload)))) return null;
205202 try {
206203 const v = JSON.parse(new TextDecoder().decode(fromB64url(payload))) as { a?: unknown; e?: unknown; x?: unknown };
207204 if (typeof v.a !== "string" || typeof v.e !== "string" || typeof v.x !== "number" || v.x < nowMs) return null;
modifiedapps/diffinery/src/mail/unsubscribe.ts+10−13View file
@@ -1,13 +1,15 @@
11 import { devAuth } from "../auth/dev.js";
22 import type { Env } from "../env.js";
3-import { b64url, fromB64url, safeEqual } from "../lib/util.js";
3+import { hmacSha256, matchesAnyKey, signingSecrets } from "../lib/crypto.js";
4+import { b64url, fromB64url } from "../lib/util.js";
45 import type { Category } from "./types.js";
56
67 /**
78 * Unsubscribe links work without signing in, so each one carries a signed
89 * capability: whose settings to change and what to turn off. The key is
9- * derived from MASTER_KEY, so links stay valid across deploys, and each one
10- * expires about a year after it was sent.
10+ * derived from MASTER_KEY, so links stay valid across deploys (and across a
11+ * rotation, while the old key is MASTER_KEY_PREV), and each one expires about
12+ * a year after it was sent.
1113 */
1214 export type UnsubScope =
1315 /** Stop email for a settings group. */
@@ -22,17 +24,12 @@
2224 scope: UnsubScope;
2325 }
2426
25-async function key(env: Env): Promise<CryptoKey> {
27+function secrets(env: Env): string[] {
2628 // Local development has no MASTER_KEY unless one is passed; production always does.
27- const secret = env.MASTER_KEY ?? (devAuth(env) ? "diffinery-dev-unsubscribe" : null);
28- if (!secret) throw new Error("MASTER_KEY is not set, so unsubscribe links can't be signed");
29- return crypto.subtle.importKey("raw", new TextEncoder().encode(`diffinery-unsubscribe-v1:${secret}`), { name: "HMAC", hash: "SHA-256" }, false, ["sign"]);
29+ return signingSecrets(env, devAuth(env) ? "diffinery-dev-unsubscribe" : null, "MASTER_KEY is not set, so unsubscribe links can't be signed");
3030 }
3131
32-async function sign(env: Env, payload: string): Promise<string> {
33- const sig = await crypto.subtle.sign("HMAC", await key(env), new TextEncoder().encode(payload));
34- return b64url(new Uint8Array(sig));
35-}
32+const signWith = async (secret: string, payload: string): Promise<string> => b64url(await hmacSha256(`diffinery-unsubscribe-v1:${secret}`, payload));
3633
3734 /** How long an unsubscribe link works: about a year, so old emails in someone's inbox stop being a capability. */
3835 export const UNSUB_TTL_MS = 365 * 24 * 60 * 60 * 1000;
@@ -42,13 +39,13 @@
4239 export async function unsubToken(env: Env, t: UnsubToken, now = Date.now()): Promise<string> {
4340 // `e` is the expiry in seconds since the epoch.
4441 const payload = b64url(new TextEncoder().encode(JSON.stringify({ a: t.accountId, s: t.scope, e: Math.floor((now + UNSUB_TTL_MS) / 1000) })));
45- return `${payload}.${await sign(env, payload)}`;
42+ return `${payload}.${await signWith(secrets(env)[0]!, payload)}`;
4643 }
4744
4845 export async function readUnsubToken(env: Env, token: string, now = Date.now()): Promise<UnsubToken | null> {
4946 const [payload, sig] = token.split(".");
5047 if (!payload || !sig || token.length > 2000) return null;
51- if (!safeEqual(sig, await sign(env, payload))) return null;
48+ if (!(await matchesAnyKey(secrets(env), sig, (s) => signWith(s, payload)))) return null;
5249 try {
5350 const v = JSON.parse(new TextDecoder().decode(fromB64url(payload))) as { a?: unknown; s?: UnsubScope; e?: unknown };
5451 if (typeof v.a !== "string" || !v.s || typeof v.s !== "object") return null;
modifiedapps/diffinery/src/repo/actions.ts+4−4View file
@@ -2137,7 +2137,7 @@
21372137 const origin = new URL(this.env.PUBLIC_URL).origin;
21382138 const exp = Math.floor(Date.now() / 1000) + (job.timeout_minutes + 30) * 60;
21392139 const { scopes } = this.repo.runtime.job(job.id);
2140- const token = await signRuntimeToken(this.env.MASTER_KEY, origin, { repo: this.repo.meta.repoId, run: run.id, job: job.id, exp }, scopes);
2140+ const token = await signRuntimeToken(this.env, origin, { repo: this.repo.meta.repoId, run: run.id, job: job.id, exp }, scopes);
21412141 return { token, resultsUrl: `${origin}/`, cacheUrl: `${origin}/_runtime/`, runtimeUrl: `${origin}/_runtime/` };
21422142 }
21432143
@@ -2322,7 +2322,7 @@
23222322 private async repoSecrets(): Promise<Record<string, string>> {
23232323 const out: Record<string, string> = {};
23242324 for (const r of this.sql.exec<{ name: string; ciphertext: string }>("SELECT name, ciphertext FROM secrets").toArray()) {
2325- out[r.name] = await decryptSecret(this.env.MASTER_KEY, `repo:${this.repo.meta.repoId}`, r.ciphertext);
2325+ out[r.name] = await decryptSecret(this.env, `repo:${this.repo.meta.repoId}`, r.ciphertext);
23262326 }
23272327 return out;
23282328 }
@@ -2339,13 +2339,13 @@
23392339 .bind(m.ownerId, m.repoId)
23402340 .all<{ name: string; ciphertext: string }>();
23412341 const out: Record<string, string> = {};
2342- for (const r of rows.results) out[r.name] = await decryptSecret(this.env.MASTER_KEY, `org:${m.ownerId}`, r.ciphertext);
2342+ for (const r of rows.results) out[r.name] = await decryptSecret(this.env, `org:${m.ownerId}`, r.ciphertext);
23432343 return out;
23442344 }
23452345
23462346 async setSecret(name: string, value: string): Promise<void> {
23472347 if (!/^[A-Za-z_][A-Za-z0-9_]*$/.test(name) || /^GITHUB_/i.test(name)) throw new HttpError(422, "invalid secret name");
2348- this.sql.exec("INSERT OR REPLACE INTO secrets (name, ciphertext, updated_at) VALUES (?, ?, ?)", name.toUpperCase(), await encryptSecret(this.env.MASTER_KEY, `repo:${this.repo.meta.repoId}`, value), now());
2348+ this.sql.exec("INSERT OR REPLACE INTO secrets (name, ciphertext, updated_at) VALUES (?, ?, ?)", name.toUpperCase(), await encryptSecret(this.env, `repo:${this.repo.meta.repoId}`, value), now());
23492349 }
23502350
23512351 deleteSecret(name: string): void {
modifiedapps/diffinery/src/repo/collab.ts+14−6View file
@@ -1,7 +1,7 @@
11 import { ZERO_OID, isAncestor, mergeBase, mergeTrees, readCommit, serializeCommit, walkCommits } from "@diffinery/git-core";
22 import { checkPublicUrl } from "../imports/options.js";
33 import type { Env } from "../env.js";
4-import { decryptSecret, encryptSecret } from "../lib/crypto.js";
4+import { encryptSecret, openSecret } from "../lib/crypto.js";
55 import type { QueueEntry } from "./queue.js";
66 import { HttpError, checkText, newId, now, randomToken } from "../lib/util.js";
77 import type { RepoDO } from "./RepoDO.js";
@@ -1073,7 +1073,8 @@
10731073 /**
10741074 * The active webhooks with their signing secrets, for delivery. Secrets are stored encrypted; a hook
10751075 * saved before that (secret_encrypted = 0) is still signed with its plaintext secret, which is
1076- * encrypted in place here so it doesn't stay in the clear.
1076+ * encrypted in place here so it doesn't stay in the clear. One sealed with MASTER_KEY_PREV, or in the
1077+ * older v1 envelope, is sealed again with the current MASTER_KEY the same way.
10771078 */
10781079 async deliverableWebhooks(): Promise<Array<Webhook & { secret: string }>> {
10791080 const rows = new Map(
@@ -1088,11 +1089,18 @@
10881089 if (!r) continue;
10891090 if (r.secret_encrypted) {
10901091 // A secret that doesn't decrypt (a changed MASTER_KEY, a damaged row) leaves that one hook undelivered, not every hook.
1091- const secret = await decryptSecret(this.env.MASTER_KEY, this.hookScope(h.id), r.secret).catch((e: Error) => void console.error("webhook secret unreadable", h.id, e.message));
1092- if (secret !== undefined) out.push({ ...h, secret });
1092+ const opened = await openSecret(this.env, this.hookScope(h.id), r.secret).catch((e: Error) => void console.error("webhook secret unreadable", h.id, e.message));
1093+ if (!opened) continue;
1094+ if (!opened.current) {
1095+ // Only if the row still holds the envelope read above; a failure here leaves it for the next delivery.
1096+ await encryptSecret(this.env, this.hookScope(h.id), opened.plaintext)
1097+ .then((sealed) => void this.sql.exec("UPDATE webhooks SET secret = ? WHERE id = ? AND secret_encrypted = 1 AND secret = ?", sealed, h.id, r.secret))
1098+ .catch((e: Error) => console.error("webhook secret not sealed again", h.id, e.message));
1099+ }
1100+ out.push({ ...h, secret: opened.plaintext });
10931101 continue;
10941102 }
1095- const sealed = await encryptSecret(this.env.MASTER_KEY, this.hookScope(h.id), r.secret);
1103+ const sealed = await encryptSecret(this.env, this.hookScope(h.id), r.secret);
10961104 // Only if the row is still the plaintext read above (no concurrent re-encryption or delete).
10971105 this.sql.exec("UPDATE webhooks SET secret = ?, secret_encrypted = 1 WHERE id = ? AND secret_encrypted = 0 AND secret = ?", sealed, h.id, r.secret);
10981106 out.push({ ...h, secret: r.secret });
@@ -1113,7 +1121,7 @@
11131121 if (count >= MAX_WEBHOOKS) throw new HttpError(422, `a repository can have at most ${MAX_WEBHOOKS} webhooks`);
11141122 const id = newId();
11151123 const s = secret ?? randomToken(24);
1116- const sealed = await encryptSecret(this.env.MASTER_KEY, this.hookScope(id), s).catch((e: Error) => {
1124+ const sealed = await encryptSecret(this.env, this.hookScope(id), s).catch((e: Error) => {
11171125 throw new HttpError(422, `could not store the webhook secret: ${e.message || "encryption unavailable"} (is MASTER_KEY configured?)`);
11181126 });
11191127 // Counted again after the await, so concurrent creates can't pass the limit together.
modifiedapps/diffinery/src/repo/deployments.ts+2−2View file
@@ -503,7 +503,7 @@
503503 const env = this.environment(envName);
504504 // An environment holding secrets defaults to protected branches only, like a fresh GitHub environment should be.
505505 if (!env.policy) this.sql.exec("UPDATE environments SET branch_policy = ?, updated_at = ? WHERE id = ?", JSON.stringify({ protected_branches: true, custom_branch_policies: false, patterns: [] } satisfies BranchPolicy), now(), env.id);
506- this.sql.exec("INSERT OR REPLACE INTO environment_secrets (environment, name, ciphertext, updated_at) VALUES (?, ?, ?, ?)", env.name, name.toUpperCase(), await encryptSecret(this.env.MASTER_KEY, this.scope(env.name), value), now());
506+ this.sql.exec("INSERT OR REPLACE INTO environment_secrets (environment, name, ciphertext, updated_at) VALUES (?, ?, ?, ?)", env.name, name.toUpperCase(), await encryptSecret(this.env, this.scope(env.name), value), now());
507507 }
508508
509509 deleteSecret(envName: string, name: string, caller: Caller): void {
@@ -610,7 +610,7 @@
610610 const secrets: Record<string, string> = {};
611611 const rows = this.secretsAllowed(name, job.ref, job.actor.kind) ? this.sql.exec<{ name: string; ciphertext: string }>("SELECT name, ciphertext FROM environment_secrets WHERE environment = ?", name).toArray() : [];
612612 for (const r of rows) {
613- secrets[r.name] = await decryptSecret(this.env.MASTER_KEY, this.scope(name), r.ciphertext);
613+ secrets[r.name] = await decryptSecret(this.env, this.scope(name), r.ciphertext);
614614 }
615615 const vars = Object.fromEntries(this.sql.exec<{ name: string; value: string }>("SELECT name, value FROM environment_variables WHERE environment = ?", name).toArray().map((r) => [r.name, r.value]));
616616 return { secrets, vars };
modifiedapps/diffinery/test/unit/actions-runtime.test.ts+2−2View file
@@ -15,8 +15,8 @@
1515 import type { Caller, RepoMeta } from "../../src/repo/types.js";
1616 import { numericId } from "../../src/http/gh.js";
1717
18-const MASTER = b64(new Uint8Array(32).fill(7));
19-const OTHER = b64(new Uint8Array(32).fill(9));
18+const MASTER = { MASTER_KEY: b64(new Uint8Array(32).fill(7)) };
19+const OTHER = { MASTER_KEY: b64(new Uint8Array(32).fill(9)) };
2020 const soon = () => Math.floor(Date.now() / 1000) + 600;
2121
2222 describe("runtime tokens", () => {
addedapps/diffinery/test/unit/master-key.test.ts+226−0View file
@@ -1,0 +1,226 @@
1+import { createHash, createHmac } from "node:crypto";
2+import { afterEach, describe, expect, it, vi } from "vitest";
3+import { decryptSecret, encryptSecret, keyId, masterKeys, matchesAnyKey, openSecret, resealSecret } from "../../src/lib/crypto.js";
4+import { signBlobUrl, signRuntimeToken, verifyBlobUrl, verifyRuntimeToken } from "../../src/lib/runtime-auth.js";
5+import { inviteProof, validInviteProof } from "../../src/auth/invite-proof.js";
6+import { readUnsubToken, unsubToken } from "../../src/mail/unsubscribe.js";
7+import { readVerifyToken, verifyToken } from "../../src/mail/suppression.js";
8+import { codeHash, codeMatches } from "../../src/auth/signup.js";
9+import { ImportService } from "../../src/imports/service.js";
10+import type { Env } from "../../src/env.js";
11+
12+const OLD = Buffer.alloc(32, 1).toString("base64");
13+const NEW = Buffer.alloc(32, 2).toString("base64");
14+const OTHER = Buffer.alloc(32, 3).toString("base64");
15+const PUB = "https://diffinery.example";
16+/** Before the rotation, during it (old key kept as MASTER_KEY_PREV), after it, and an unrelated instance. */
17+const before = { MASTER_KEY: OLD, PUBLIC_URL: PUB } as unknown as Env;
18+const during = { MASTER_KEY: NEW, MASTER_KEY_PREV: OLD, PUBLIC_URL: PUB } as unknown as Env;
19+const after = { MASTER_KEY: NEW, PUBLIC_URL: PUB } as unknown as Env;
20+const other = { MASTER_KEY: OTHER, PUBLIC_URL: PUB } as unknown as Env;
21+
22+const kidOf = (k: string) => createHash("sha256").update(Buffer.from(k, "base64")).digest("hex").slice(0, 8);
23+const hmacB64url = (key: string, data: string) => createHmac("sha256", key).update(data).digest("base64url");
24+
25+describe("master keys", () => {
26+ it("are the current key first, then the previous one while rotating", () => {
27+ expect(masterKeys(before)).toEqual([OLD]);
28+ expect(masterKeys(during)).toEqual([NEW, OLD]);
29+ // The same value twice, or an empty one, counts once; MASTER_KEY_PREV alone is no key at all.
30+ expect(masterKeys({ MASTER_KEY: OLD, MASTER_KEY_PREV: OLD })).toEqual([OLD]);
31+ expect(masterKeys({ MASTER_KEY: OLD, MASTER_KEY_PREV: "" })).toEqual([OLD]);
32+ expect(masterKeys({ MASTER_KEY_PREV: OLD })).toEqual([]);
33+ });
34+
35+ it("are named in envelopes by the first 8 hex characters of SHA-256 over their bytes", async () => {
36+ expect(await keyId(OLD)).toBe(kidOf(OLD));
37+ expect(await keyId(NEW)).not.toBe(await keyId(OLD));
38+ });
39+
40+ it("a malformed previous key never breaks a check under the current one", async () => {
41+ const mac = async (k: string) => Buffer.from(k, "base64").toString("hex");
42+ expect(await matchesAnyKey([NEW, "!!"], await mac(NEW), mac)).toBe(true);
43+ await expect(matchesAnyKey(["!!", NEW], "x", async () => Promise.reject(new Error("bad key")))).rejects.toThrow("bad key");
44+ });
45+});
46+
47+describe("secret envelopes", () => {
48+ // Written by the v1 code (before keys could be rotated) with OLD, scope repo:r1.
49+ const V1 = "v1.u9iC/IUkKT+24mVm.pCaoBWOPvxHGC6vK6GkimNm7zhfgouc=";
50+
51+ it("written before rotation existed still decrypt, before and during a rotation", async () => {
52+ expect(await decryptSecret(before, "repo:r1", V1)).toBe("hunter2");
53+ expect(await openSecret(during, "repo:r1", V1)).toEqual({ plaintext: "hunter2", current: false });
54+ expect(await openSecret(before, "repo:r1", V1)).toEqual({ plaintext: "hunter2", current: false });
55+ });
56+
57+ it("are v2 with the key id now, and round-trip", async () => {
58+ const e = await encryptSecret(before, "repo:r1", "hunter2");
59+ expect(e).toMatch(new RegExp(`^v2\\.${kidOf(OLD)}\\.[A-Za-z0-9+/=]+\\.[A-Za-z0-9+/=]+$`));
60+ expect(e).not.toContain("hunter2");
61+ expect(await openSecret(before, "repo:r1", e)).toEqual({ plaintext: "hunter2", current: true });
62+ // The same AES-GCM ciphertext as v1 would carry: dropping the key id gives a valid v1 envelope.
63+ const [, , iv, ct] = e.split(".");
64+ expect(await decryptSecret(before, "repo:r1", `v1.${iv}.${ct}`)).toBe("hunter2");
65+ });
66+
67+ it("sealed with the old key decrypt with MASTER_KEY_PREV, and new ones use the new key", async () => {
68+ const old = await encryptSecret(before, "org:o1", "s3cret");
69+ expect(await openSecret(during, "org:o1", old)).toEqual({ plaintext: "s3cret", current: false });
70+ const fresh = await encryptSecret(during, "org:o1", "s3cret");
71+ expect(fresh.split(".")[1]).toBe(kidOf(NEW));
72+ expect(await openSecret(during, "org:o1", fresh)).toEqual({ plaintext: "s3cret", current: true });
73+ expect(await decryptSecret(after, "org:o1", fresh)).toBe("s3cret");
74+ // Once MASTER_KEY_PREV is gone, what only the old key opens doesn't decrypt.
75+ await expect(decryptSecret(after, "org:o1", old)).rejects.toThrow(/doesn't decrypt with MASTER_KEY$/);
76+ await expect(decryptSecret(after, "repo:r1", V1)).rejects.toThrow();
77+ });
78+
79+ it("are tried with every key when the key id names none of them", async () => {
80+ const e = await encryptSecret(before, "repo:r1", "x");
81+ const [v, , iv, ct] = e.split(".");
82+ expect(await openSecret(during, "repo:r1", `${v}.00000000.${iv}.${ct}`)).toEqual({ plaintext: "x", current: false });
83+ });
84+
85+ it("don't decrypt with a wrong key, in another scope, or when damaged", async () => {
86+ const e = await encryptSecret(before, "repo:r1", "hunter2");
87+ await expect(decryptSecret(other, "repo:r1", e)).rejects.toThrow(/doesn't decrypt/);
88+ await expect(decryptSecret({ MASTER_KEY: OTHER, MASTER_KEY_PREV: NEW }, "repo:r1", e)).rejects.toThrow(/MASTER_KEY or MASTER_KEY_PREV/);
89+ await expect(decryptSecret(before, "repo:r2", e)).rejects.toThrow(/doesn't decrypt/);
90+ await expect(decryptSecret(before, "repo:r1", V1.replace("pCa", "pCb"))).rejects.toThrow(/doesn't decrypt/);
91+ for (const bad of ["v3.a.b", "v1.a", "v2.zzzzzzzz.a.b", "v2.a.b", "plain"]) await expect(decryptSecret(before, "repo:r1", bad), bad).rejects.toThrow(/bad secret envelope/);
92+ await expect(encryptSecret({ MASTER_KEY_PREV: OLD }, "repo:r1", "x")).rejects.toThrow(/MASTER_KEY is not configured/);
93+ await expect(decryptSecret({}, "repo:r1", V1)).rejects.toThrow(/MASTER_KEY is not configured/);
94+ });
95+
96+ it("are sealed again only when that is worth it, and never lost", async () => {
97+ const current = await encryptSecret(during, "repo:r1", "hunter2");
98+ expect(await resealSecret(during, "repo:r1", current)).toBe(current);
99+ for (const stale of [V1, await encryptSecret(before, "repo:r1", "hunter2")]) {
100+ const resealed = await resealSecret(during, "repo:r1", stale);
101+ expect(resealed.split(".").slice(0, 2)).toEqual(["v2", kidOf(NEW)]);
102+ expect(await decryptSecret(after, "repo:r1", resealed)).toBe("hunter2");
103+ }
104+ // One that doesn't decrypt is kept as it was.
105+ expect(await resealSecret(other, "repo:r1", V1)).toBe(V1);
106+ });
107+});
108+
109+describe("signatures made with MASTER_KEY", () => {
110+ it("runtime tokens: old ones verify while rotating, new ones use the new key", async () => {
111+ const exp = Math.floor(Date.now() / 1000) + 600;
112+ const claims = { repo: "r1", run: "run1", job: "job1", exp };
113+ const old = await signRuntimeToken(before, "x", claims, ["refs/heads/main"]);
114+ expect(await verifyRuntimeToken(during, old)).toMatchObject({ repo: "r1", job: "job1" });
115+ expect(await verifyRuntimeToken(after, old)).toBeNull();
116+ const fresh = await signRuntimeToken(during, "x", claims, ["refs/heads/main"]);
117+ expect(await verifyRuntimeToken(after, fresh)).toMatchObject({ repo: "r1" });
118+ expect(await verifyRuntimeToken(before, fresh)).toBeNull();
119+ expect(await verifyRuntimeToken(other, old)).toBeNull();
120+ expect(await verifyRuntimeToken({}, old)).toBeNull();
121+ });
122+
123+ it("blob URLs: signed exactly as before, accepted with MASTER_KEY_PREV, new ones under the new key", async () => {
124+ const ref = { repo: "r1", kind: "artifact" as const, id: 42 };
125+ const old = new URL(await signBlobUrl(before, "https://d.example", ref, "r", 4102444800));
126+ // Signed by the code before rotation existed, with OLD.
127+ expect(old.searchParams.get("sig")).toBe("1I0_HL80HFt73S7qB9xv13q82j8Y8OvdubCBDKmlufM");
128+ expect(await verifyBlobUrl(during, ref, "r", old.searchParams)).toBe(true);
129+ expect(await verifyBlobUrl(after, ref, "r", old.searchParams)).toBe(false);
130+ const fresh = new URL(await signBlobUrl(during, "https://d.example", ref, "r", 4102444800));
131+ expect(fresh.searchParams.get("sig")).not.toBe(old.searchParams.get("sig"));
132+ expect(await verifyBlobUrl(after, ref, "r", fresh.searchParams)).toBe(true);
133+ expect(await verifyBlobUrl(other, ref, "r", old.searchParams)).toBe(false);
134+ });
135+
136+ it("invitation proofs", async () => {
137+ const old = await inviteProof(before, "hash1");
138+ expect(old).toBe(hmacB64url(`diffinery-invite-proof-v1:${OLD}`, "hash1"));
139+ expect(await validInviteProof(during, "hash1", old)).toBe(true);
140+ expect(await validInviteProof(during, "hash2", old)).toBe(false);
141+ expect(await validInviteProof(after, "hash1", old)).toBe(false);
142+ const fresh = await inviteProof(during, "hash1");
143+ expect(fresh).toBe(hmacB64url(`diffinery-invite-proof-v1:${NEW}`, "hash1"));
144+ expect(await validInviteProof(after, "hash1", fresh)).toBe(true);
145+ expect(await validInviteProof(other, "hash1", old)).toBe(false);
146+ });
147+
148+ it("unsubscribe links", async () => {
149+ const t = { accountId: "acc1", scope: { t: "category" as const, category: "watching" as const } };
150+ const old = await unsubToken(before, t);
151+ const [payload, sig] = old.split(".");
152+ expect(sig).toBe(hmacB64url(`diffinery-unsubscribe-v1:${OLD}`, payload!));
153+ expect(await readUnsubToken(during, old)).toEqual(t);
154+ expect(await readUnsubToken(after, old)).toBeNull();
155+ const fresh = await unsubToken(during, t);
156+ expect(fresh.split(".")[1]).toBe(hmacB64url(`diffinery-unsubscribe-v1:${NEW}`, fresh.split(".")[0]!));
157+ expect(await readUnsubToken(after, fresh)).toEqual(t);
158+ expect(await readUnsubToken(other, old)).toBeNull();
159+ });
160+
161+ it("email verification links", async () => {
162+ const t = { accountId: "ann", email: "ann@example.com", expiresAt: Date.now() + 60_000 };
163+ const old = await verifyToken(before, t);
164+ expect(old.split(".")[1]).toBe(hmacB64url(`diffinery-verify-email-v1:${OLD}`, old.split(".")[0]!));
165+ expect(await readVerifyToken(during, old)).toMatchObject({ accountId: "ann" });
166+ expect(await readVerifyToken(after, old)).toBeNull();
167+ const fresh = await verifyToken(during, t);
168+ expect(await readVerifyToken(after, fresh)).toMatchObject({ accountId: "ann" });
169+ expect(await readVerifyToken(other, old)).toBeNull();
170+ });
171+
172+ it("sign-up code hashes", async () => {
173+ const stored = await codeHash(before, "row1", "123456");
174+ expect(stored).toBe(hmacB64url(`diffinery-signup-code-v1:${OLD}`, "row1:123456"));
175+ expect(await codeMatches(before, "row1", "123456", stored)).toBe(true);
176+ expect(await codeMatches(during, "row1", "123456", stored)).toBe(true);
177+ expect(await codeMatches(during, "row1", "654321", stored)).toBe(false);
178+ expect(await codeMatches(after, "row1", "123456", stored)).toBe(false);
179+ expect(await codeHash(during, "row1", "123456")).toBe(hmacB64url(`diffinery-signup-code-v1:${NEW}`, "row1:123456"));
180+ expect(await codeMatches(other, "row1", "123456", stored)).toBe(false);
181+ });
182+
183+ describe("GitHub account links", () => {
184+ afterEach(() => vi.unstubAllGlobals());
185+ const hex16 = (key: string, data: string) => createHmac("sha256", key).update(data).digest("hex").slice(0, 32);
186+ const svc = (env: Env) => new ImportService({ ...env, DB: {}, GITHUB_OAUTH_CLIENT_ID: "id", GITHUB_OAUTH_CLIENT_SECRET: "s" } as unknown as Env);
187+ const person = { id: "u1", kind: "user", login: "ann" } as never;
188+
189+ it("show the gist nonce under MASTER_KEY, and also take the one under MASTER_KEY_PREV", async () => {
190+ const api = "https://api.github.com";
191+ const { nonce } = await svc(before).linkChallenge(person, null);
192+ expect(nonce).toBe(`diffinery-link-${hex16(OLD, `github-link:u1:${api}`)}`);
193+ const nonces = await (svc(during) as unknown as { linkNonces(a: string, b: string): Promise<string[]> }).linkNonces("u1", api);
194+ expect(nonces).toEqual([`diffinery-link-${hex16(NEW, `github-link:u1:${api}`)}`, nonce]);
195+ expect((await svc(during).linkChallenge(person, null)).nonce).toBe(nonces[0]);
196+ });
197+
198+ it("accept an OAuth state made with MASTER_KEY_PREV", async () => {
199+ const state = new URL(await svc(before).oauthStart(person)).searchParams.get("state")!;
200+ expect(state).toBe(`u1.${hex16(OLD, "github-oauth:u1")}`);
201+ // Past the state check, GitHub is asked for a token (and here returns none).
202+ vi.stubGlobal("fetch", async () => new Response("{}"));
203+ await expect(svc(during).oauthCallback(person, "code", state)).rejects.toMatchObject({ status: 422, detail: expect.stringMatching(/access token/) });
204+ await expect(svc(after).oauthCallback(person, "code", state)).rejects.toMatchObject({ status: 422, detail: expect.stringMatching(/OAuth state/) });
205+ await expect(svc(during).oauthCallback(person, "code", `u1.${hex16(OTHER, "github-oauth:u1")}`)).rejects.toMatchObject({ status: 422, detail: expect.stringMatching(/OAuth state/) });
206+ });
207+ });
208+});
209+
210+describe("sealing again with the current key", () => {
211+ const person = { id: "u1", kind: "user", login: "ann" } as never;
212+
213+ it("a retried import seals the token it keeps with the new key", async () => {
214+ const token = await encryptSecret(before, "import:imp-1", "ghp_x");
215+ const row = { id: "imp-1", parent_id: null, kind: "repo", status: "failed", created_by: "u1", target_owner_id: "u1", repo_id: null, attempt: 1, token_ciphertext: token, source: "octo/app" };
216+ const writes: unknown[][] = [];
217+ const db = { prepare: () => ({ bind: (...a: unknown[]) => ({ run: async () => void writes.push(a), first: async () => ({ n: 0 }) }) }) };
218+ const s = new ImportService({ ...during, DB: db, IMPORT_WORKFLOW: { create: async () => ({}) } } as unknown as Env);
219+ Object.assign(s, { row: async () => row, audit: async () => undefined, canImportInto: async () => true });
220+ Object.assign(s.dir, { account: async () => ({ id: "u1", kind: "user", login: "ann" }) });
221+ await s.retry("imp-1", person, null);
222+ const stored = writes.map((w) => w[1]).find((v) => typeof v === "string" && v.startsWith("v2.")) as string;
223+ expect(stored.split(".")[1]).toBe(kidOf(NEW));
224+ expect(await decryptSecret(after, "import:imp-1", stored)).toBe("ghp_x");
225+ });
226+});
modifiedapps/diffinery/test/unit/webhooks.test.ts+20−1View file
@@ -2,6 +2,7 @@
22 import { describe, expect, it } from "vitest";
33 import type { Env } from "../../src/env.js";
44 import { Collab } from "../../src/repo/collab.js";
5+import { encryptSecret } from "../../src/lib/crypto.js";
56 import type { RepoDO } from "../../src/repo/RepoDO.js";
67 import { REPO_SCHEMA, migrateRepoSchema } from "../../src/repo/schema.js";
78 import { deliverWebhook, WEBHOOK_RETRY_DELAYS, type WebhookDeps, type WebhookRetry } from "../../src/lib/webhooks.js";
@@ -111,7 +112,7 @@
111112 expect(made.secret).toBe("s3cret");
112113 const row = stored(sql, made.id);
113114 expect(row.secret_encrypted).toBe(1);
114- expect(row.secret).toMatch(/^v1\./);
115+ expect(row.secret).toMatch(/^v2\.[0-9a-f]{8}\./);
115116 expect(row.secret).not.toContain("s3cret");
116117 expect(c.webhooks()[0]).not.toHaveProperty("secret");
117118 expect((await c.deliverableWebhooks()).map((h) => h.secret)).toEqual(["s3cret"]);
@@ -141,6 +142,24 @@
141142 expect((await c.deliverableWebhooks()).map((h) => h.secret)).toEqual(["v1.not.encrypted"]);
142143 });
143144
145+ it("sealed with the previous MASTER_KEY, or as v1, are sealed again with the current one on delivery", async () => {
146+ const sql = sqlStorage();
147+ const next = { MASTER_KEY: Buffer.alloc(32, 8).toString("base64"), MASTER_KEY_PREV: env.MASTER_KEY } as Env;
148+ const before = new Collab({ sql, meta: { repoId: "r1" } } as unknown as RepoDO, env);
149+ const a = await before.createWebhook("https://hooks.example/a", [], "aaa", "diffinery.test");
150+ const b = await before.createWebhook("https://hooks.example/b", [], "bbb", "diffinery.test");
151+ // b as the v1 code wrote it: the same ciphertext without a key id.
152+ const [, , iv, ct] = stored(sql, b.id).secret.split(".");
153+ sql.exec("UPDATE webhooks SET secret = ? WHERE id = ?", `v1.${iv}.${ct}`, b.id);
154+ const kid = (await encryptSecret(next, "x", "x")).split(".")[1];
155+ const during = new Collab({ sql, meta: { repoId: "r1" } } as unknown as RepoDO, next);
156+ expect((await during.deliverableWebhooks()).map((h) => h.secret).sort()).toEqual(["aaa", "bbb"]);
157+ for (const id of [a.id, b.id]) expect(stored(sql, id).secret.split(".").slice(0, 2)).toEqual(["v2", kid]);
158+ // After the rotation, without the previous key.
159+ const after = new Collab({ sql, meta: { repoId: "r1" } } as unknown as RepoDO, { MASTER_KEY: next.MASTER_KEY } as Env);
160+ expect((await after.deliverableWebhooks()).map((h) => h.secret).sort()).toEqual(["aaa", "bbb"]);
161+ });
162+
144163 it("leave inactive hooks out of delivery", async () => {
145164 const { sql, collab: c } = collab();
146165 const h = await c.createWebhook("https://hooks.example/x", [], "s", "diffinery.test");
modifiedscripts/ensure-secrets.sh+3−1View file
@@ -1,6 +1,8 @@
11 #!/usr/bin/env bash
22 # Generate the Worker's own secrets once (never rotated automatically):
3-# MASTER_KEY root key for encrypting Actions secrets
3+# MASTER_KEY root key for encrypting Actions secrets (rotated by hand, with
4+# MASTER_KEY_PREV, which this script never touches; see
5+# apps/diffinery/docs/self-hosting/configuration.md, "Rotating MASTER_KEY")
46 # OIDC_PRIVATE_KEY ES256 key that signs workflow OIDC id tokens
57 # and, when missing, sets TURNSTILE_SECRET from the Turnstile widget whose site key was deployed
68 # (scripts/turnstile.sh, through bootstrap and render-config).