5xx answers carry a request ID that finds their error under Settings → Server errors; the sync retries its secret PUTs (#143)
Every 5xx answer is now recorded in D1 (server_errors, migration 0024): request ID, method, route, path, status, message and stack. Tokens are masked; no request bodies, query strings or headers are kept; only the newest 500 records are kept. The JSON answer carries request_id and an x-request-id header, and error pages show the ID. Site administrators read the records at GET /api/v3/diffinery/admin/errors[/{id}] or in Settings → Server errors.
The GitHub-to-Diffinery sync retries its idempotent secret and variable PUTs, and prints the answer when one fails.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ruyx3Qr4jYrZEpPi466debCanberk Demirci authored · GitHub committed signed1 parent 2152c82commit 4a91970836ce2a4a3db6db0ad0b82ec412e898eaBrowse files
11 files changed+321−11
Files (11)
- .github/workflows/sync-to-diffinery.yml+5−1
- apps/diffinery/docs/admin/site-administration.md+11−0
- apps/diffinery/migrations/0024_server_errors.sql+14−0
- apps/diffinery/src/app.ts+1−1
- apps/diffinery/src/docs/api-meta.ts+7−0
- apps/diffinery/src/http/api.ts+13−0
- apps/diffinery/src/http/common.ts+36−6
- apps/diffinery/src/lib/server-errors.ts+72−0
- apps/diffinery/src/web/account.tsx+86−2
- apps/diffinery/src/web/ui.tsx+8−1
- apps/diffinery/test/unit/server-errors.test.ts+68−0
modified.github/workflows/sync-to-diffinery.yml+5−1View file
| @@ -77,7 +77,11 @@ | ||
| 77 | 77 | V_MAIL_DOMAIN: ${{ vars.MAIL_DOMAIN }} |
| 78 | 78 | run: | |
| 79 | 79 | set -euo pipefail |
| 80 | - put() { curl -fsS -o /dev/null -X PUT "$DIFFINERY/api/v3/repos/$ORG/$NAME/actions/$1/$2" -H "Authorization: Bearer $TOKEN" -H "content-type: application/json" --data-binary @-; } | |
| 80 | + # The PUT is idempotent, so a transient 5xx is retried; a failure shows the answer (its request_id finds the error under Settings -> Server errors). | |
| 81 | + put() { | |
| 82 | + local out | |
| 83 | + out=$(curl -sS --fail-with-body --retry 3 --retry-all-errors --retry-delay 2 -X PUT "$DIFFINERY/api/v3/repos/$ORG/$NAME/actions/$1/$2" -H "Authorization: Bearer $TOKEN" -H "content-type: application/json" --data-binary @-) || { echo "::error::PUT actions/$1/$2 failed: $out"; return 1; } | |
| 84 | + } | |
| 81 | 85 | for name in CLOUDFLARE_API_TOKEN CLOUDFLARE_ACCOUNT_ID DIFFINERY_LIVE_TOKEN IMPORT_GITHUB_TOKEN; do |
| 82 | 86 | v="S_$name" |
| 83 | 87 | if [ -n "${!v}" ]; then jq -n --arg v "${!v}" '{value: $v}' | put secrets "$name"; echo "secret $name copied"; else echo "::warning::secret $name is empty here, so it was not copied"; fi |
modifiedapps/diffinery/docs/admin/site-administration.md+11−0View file
| @@ -268,6 +268,17 @@ | ||
| 268 | 268 | |
| 269 | 269 | A `last_tick_at` far behind the time means checks are failing; look in the Worker's logs for `schedule tick failed`. Site administrators only, with any token scope. How it works is in [Architecture](/docs/self-hosting/architecture#workflow-schedules). |
| 270 | 270 | |
| 271 | +## Server errors | |
| 272 | + | |
| 273 | +When Diffinery answers a request with a 5xx status, it keeps a record of the error. The answer's JSON includes a `request_id`, which also appears in the `x-request-id` header, and an error page shows it as **Request ID**. Look the request ID up under **Settings → Server errors**, or over the API: | |
| 274 | + | |
| 275 | +```bash | |
| 276 | +diffinery api GET /diffinery/admin/errors/<request id> | |
| 277 | +diffinery api GET "/diffinery/admin/errors?per_page=20" | |
| 278 | +``` | |
| 279 | + | |
| 280 | +Each record has the request's method, `route` (the matched route pattern), `path`, `status`, `message` and `stack`. Only the newest 500 errors are kept. Records never include request bodies, query strings or headers. Tokens that appear in a message are masked, and a path that carries a secret, such as a sign-in link, is stored as its route pattern instead. On Cloudflare, the request ID is the request's Ray ID, so the same request can also be found in Cloudflare's logs. | |
| 281 | + | |
| 271 | 282 | ## Administration from the CLI |
| 272 | 283 | |
| 273 | 284 | The CLI (needs Node.js 22 or later) covers invites and, through `diffinery api`, any API endpoint: |
addedapps/diffinery/migrations/0024_server_errors.sql+14−0View file
| @@ -1,0 +1,14 @@ | ||
| 1 | +-- Unexpected server errors (lib/server-errors.ts): the newest 500 answers with a 5xx status, for | |
| 2 | +-- site administrators. `id` is the request id the answer carried. No request bodies, query | |
| 3 | +-- strings or headers are kept. | |
| 4 | +CREATE TABLE IF NOT EXISTS server_errors ( | |
| 5 | + id TEXT PRIMARY KEY, | |
| 6 | + at TEXT NOT NULL, | |
| 7 | + method TEXT NOT NULL, | |
| 8 | + route TEXT, | |
| 9 | + path TEXT NOT NULL, | |
| 10 | + status INTEGER NOT NULL, | |
| 11 | + message TEXT NOT NULL, | |
| 12 | + stack TEXT | |
| 13 | +); | |
| 14 | +CREATE INDEX IF NOT EXISTS server_errors_at ON server_errors(at); |
modifiedapps/diffinery/src/app.ts+1−1View file
| @@ -35,7 +35,7 @@ | ||
| 35 | 35 | |
| 36 | 36 | export const app = new Hono<{ Bindings: Env; Variables: AppVars }>(); |
| 37 | 37 | |
| 38 | -app.onError((e) => errorResponse(e)); | |
| 38 | +app.onError((e, c) => errorResponse(e, c)); | |
| 39 | 39 | app.notFound((c) => (c.req.path.startsWith("/api/") ? apiNotFound() : c.text("404 Not Found", 404))); |
| 40 | 40 | |
| 41 | 41 | // The model proxy for agent tasks: its own task-scoped token, not an account credential (agentic/proxy.ts). |
modifiedapps/diffinery/src/docs/api-meta.ts+7−0View file
| @@ -1450,6 +1450,13 @@ | ||
| 1450 | 1450 | description: "`tick_minutes`, `last_tick_at` (when the last check finished), `covered_until` (the last minute whose schedules were started) and `next_tick_at` (null when the timer isn't set). Tells a scheduler that stopped ticking from a run that failed to start.", |
| 1451 | 1451 | access: "site-admin", |
| 1452 | 1452 | }, |
| 1453 | + "GET /diffinery/admin/errors": { | |
| 1454 | + summary: "List unexpected server errors", | |
| 1455 | + description: "The newest answers with a 5xx status, newest first: `id` (the `request_id` the answer carried, also in its `x-request-id` header), `at`, `method`, `route`, `path`, `status`, `message` and `stack`. The newest 500 are kept. No request bodies, query strings or headers are kept, and tokens in a message are masked.", | |
| 1456 | + access: "site-admin", | |
| 1457 | + query: { per_page: ["integer", "At most this many (default 100, up to 500)."] }, | |
| 1458 | + }, | |
| 1459 | + "GET /diffinery/admin/errors/{id}": { summary: "Get a server error by its request ID", description: "The error a 5xx answer with this `request_id` recorded; 404 once it is no longer among the newest 500.", access: "site-admin" }, | |
| 1453 | 1460 | "POST /diffinery/admin/emails/samples": { summary: "Send yourself every email template", access: "site-admin", body: { only: ["string[]", "Only these sample ids."] } }, |
| 1454 | 1461 | |
| 1455 | 1462 | // ------------------------------------------------------------ tokens, agents and sandboxes |
modifiedapps/diffinery/src/http/api.ts+13−0View file
| @@ -5,6 +5,7 @@ | ||
| 5 | 5 | import { baseRole, type Account, type Directory, type Repo, type TokenScope } from "../lib/directory.js"; |
| 6 | 6 | import { encryptSecret } from "../lib/crypto.js"; |
| 7 | 7 | import { monthStart, quotasFor, setQuotas, usageOf } from "../lib/quotas.js"; |
| 8 | +import { serverErrors } from "../lib/server-errors.js"; | |
| 8 | 9 | import { changePause, changeSuspension, cleanReason, computePause, killSwitches, suspensionOf, type Stopped } from "../lib/killswitch.js"; |
| 9 | 10 | import { publicJwk } from "../lib/jwt.js"; |
| 10 | 11 | import { HttpError, b64, fromB64 } from "../lib/util.js"; |
| @@ -65,6 +66,18 @@ | ||
| 65 | 66 | |
| 66 | 67 | api.get("/", (c) => c.json({ name: "Diffinery", api: "v3", docs: `${c.env.PUBLIC_URL}/docs/api`, mcp: `${c.env.PUBLIC_URL}/mcp` })); |
| 67 | 68 | api.get("/meta", (c) => c.json({ name: "Diffinery", oidc: !!c.env.OIDC_PRIVATE_KEY, runner_image: c.env.RUNNER_IMAGE ?? null })); |
| 69 | +/** Site admin: the newest unexpected server errors (lib/server-errors.ts), and one by the request id its 5xx answer carried. */ | |
| 70 | +api.get("/diffinery/admin/errors", async (c) => { | |
| 71 | + if (!requireAccount(c).site_admin) throw new HttpError(403, "site administrators only"); | |
| 72 | + const n = Number(c.req.query("per_page") ?? 100); | |
| 73 | + return c.json(await serverErrors(c.env, { limit: Number.isFinite(n) ? n : 100 })); | |
| 74 | +}); | |
| 75 | +api.get("/diffinery/admin/errors/:id", async (c) => { | |
| 76 | + if (!requireAccount(c).site_admin) throw new HttpError(403, "site administrators only"); | |
| 77 | + const [e] = await serverErrors(c.env, { id: c.req.param("id") }); | |
| 78 | + if (!e) throw new HttpError(404, "no server error with that request id (only the newest 500 are kept)"); | |
| 79 | + return c.json(e); | |
| 80 | +}); | |
| 68 | 81 | /** Site admin: the workflow scheduler's state, to tell a scheduler that stopped ticking from a run that failed to start. */ |
| 69 | 82 | api.get("/diffinery/admin/scheduler", async (c) => { |
| 70 | 83 | if (!requireAccount(c).site_admin) throw new HttpError(403, "site administrators only"); |
modifiedapps/diffinery/src/http/common.ts+36−6View file
| @@ -2,6 +2,7 @@ | ||
| 2 | 2 | import type { Repo } from "../lib/directory.js"; |
| 3 | 3 | import { denied, readOnly, requestPermission, type JobPermissions } from "../lib/job-permissions.js"; |
| 4 | 4 | import { HttpError } from "../lib/util.js"; |
| 5 | +import { errorRecord, recordServerError, requestId } from "../lib/server-errors.js"; | |
| 5 | 6 | import { movedPath, movedStatus } from "../lib/redirects.js"; |
| 6 | 7 | import { GITHUB_DOCS } from "./gh.js"; |
| 7 | 8 | import { callerFor, requireScope, type AppContext } from "../auth/principal.js"; |
| @@ -99,18 +100,47 @@ | ||
| 99 | 100 | return new Response(JSON.stringify({ message: "Not Found", documentation_url: GITHUB_DOCS }), { status: 404, headers: { "content-type": "application/json" } }); |
| 100 | 101 | } |
| 101 | 102 | |
| 102 | -export function errorResponse(e: unknown): Response { | |
| 103 | +/** | |
| 104 | + * Records a 5xx answer for site administrators (lib/server-errors.ts) and returns the request id | |
| 105 | + * the answer reports, so a failure seen in a client or CI log can be looked up. | |
| 106 | + */ | |
| 107 | +export function noteServerError(c: AppContext, e: unknown, status: number): string { | |
| 108 | + const id = requestId(c.req.raw); | |
| 109 | + let route: string | null = null; | |
| 110 | + try { | |
| 111 | + route = c.req.routePath; | |
| 112 | + } catch { | |
| 113 | + // No route matched. | |
| 114 | + } | |
| 115 | + const saved = recordServerError(c.env, errorRecord(e, { id, method: c.req.method, route, path: c.req.path, status })); | |
| 116 | + try { | |
| 117 | + c.executionCtx.waitUntil(saved); | |
| 118 | + } catch { | |
| 119 | + // No execution context (tests): the write runs on its own. | |
| 120 | + } | |
| 121 | + return id; | |
| 122 | +} | |
| 123 | + | |
| 124 | +export function errorResponse(e: unknown, c?: AppContext): Response { | |
| 125 | + const json = (status: number, body: Record<string, unknown>, headers: Record<string, string> = {}) => { | |
| 126 | + if (status >= 500 && c) { | |
| 127 | + const id = noteServerError(c, e, status); | |
| 128 | + body = { ...body, request_id: id }; | |
| 129 | + headers = { ...headers, "x-request-id": id }; | |
| 130 | + } | |
| 131 | + return new Response(JSON.stringify(body), { status, headers: { "content-type": "application/json", ...headers } }); | |
| 132 | + }; | |
| 103 | 133 | if (e instanceof HttpError) { |
| 104 | - const headers: Record<string, string> = { "content-type": "application/json" }; | |
| 134 | + const headers: Record<string, string> = {}; | |
| 105 | 135 | if (e.status === 401) headers["www-authenticate"] = 'Basic realm="Diffinery", charset="UTF-8"'; |
| 106 | 136 | // GitHub's answer for a renamed or transferred repository: the new URL in Location and in the body. |
| 107 | - if (e instanceof RepoMoved) return new Response(JSON.stringify({ message: e.detail, url: e.location, documentation_url: GITHUB_DOCS }), { status: e.status, headers: { ...headers, location: e.location } }); | |
| 108 | - return new Response(JSON.stringify({ message: e.detail, documentation_url: GITHUB_DOCS, ...e.extra }), { status: e.status, headers }); | |
| 137 | + if (e instanceof RepoMoved) return json(e.status, { message: e.detail, url: e.location, documentation_url: GITHUB_DOCS }, { ...headers, location: e.location }); | |
| 138 | + return json(e.status, { message: e.detail, documentation_url: GITHUB_DOCS, ...e.extra }, headers); | |
| 109 | 139 | } |
| 110 | 140 | // Errors thrown inside Durable Objects arrive as plain Errors; recover HttpError-like status when present. |
| 111 | 141 | const msg = (e as Error)?.message ?? String(e); |
| 112 | 142 | const remote = /^(\d{3}):/.exec(msg); |
| 113 | - if (remote) return new Response(JSON.stringify({ message: msg.slice(5), documentation_url: GITHUB_DOCS }), { status: Number(remote[1]), headers: { "content-type": "application/json" } }); | |
| 143 | + if (remote) return json(Number(remote[1]), { message: msg.slice(5), documentation_url: GITHUB_DOCS }); | |
| 114 | 144 | console.error(e); |
| 115 | - return new Response(JSON.stringify({ message: "internal error" }), { status: 500, headers: { "content-type": "application/json" } }); | |
| 145 | + return json(500, { message: "internal error" }); | |
| 116 | 146 | } |
addedapps/diffinery/src/lib/server-errors.ts+72−0View file
| @@ -1,0 +1,72 @@ | ||
| 1 | +import type { Env } from "../env.js"; | |
| 2 | + | |
| 3 | +/** | |
| 4 | + * Unexpected server errors (5xx answers), kept in D1 so site administrators can read them | |
| 5 | + * without Cloudflare log access. A 5xx answer carries its request id, which finds the record. | |
| 6 | + * Records hold the method, route, path, status, message and stack, never request bodies, | |
| 7 | + * query strings or headers; token-like strings in a message or stack are masked. | |
| 8 | + */ | |
| 9 | +export const KEEP_ERRORS = 500; | |
| 10 | + | |
| 11 | +export interface ServerError { | |
| 12 | + id: string; | |
| 13 | + at: string; | |
| 14 | + method: string; | |
| 15 | + route: string | null; | |
| 16 | + path: string; | |
| 17 | + status: number; | |
| 18 | + message: string; | |
| 19 | + stack: string | null; | |
| 20 | +} | |
| 21 | + | |
| 22 | +/** The id a 5xx answer reports: Cloudflare's ray id when there is one, so it also finds the request in Cloudflare's logs. */ | |
| 23 | +export function requestId(req: Request): string { | |
| 24 | + const ray = req.headers.get("cf-ray"); | |
| 25 | + return ray && /^[\w-]{8,64}$/.test(ray) ? ray : crypto.randomUUID(); | |
| 26 | +} | |
| 27 | + | |
| 28 | +/** Masks credentials an error message might quote: Diffinery and GitHub tokens, bearer and basic auth, and URL user info. */ | |
| 29 | +export function redact(s: string): string { | |
| 30 | + return s | |
| 31 | + .replace(/\b(dfy|ghp|gho|ghu|ghs|ghr|github_pat)_[A-Za-z0-9_]{8,}/g, "$1_…") | |
| 32 | + .replace(/\b(Bearer|Basic|token)\s+[A-Za-z0-9._~+/=-]{8,}/gi, "$1 …") | |
| 33 | + .replace(/(\w+:\/\/)[^/\s@]+@/g, "$1…@"); | |
| 34 | +} | |
| 35 | + | |
| 36 | +/** A request path to keep: a route that carries a secret in the path (`/signin/:token`) keeps its pattern instead. */ | |
| 37 | +export function pathToKeep(path: string, route: string | null): string { | |
| 38 | + return route && /:(token|code|secret|key)\b/i.test(route) ? route : path.slice(0, 500); | |
| 39 | +} | |
| 40 | + | |
| 41 | +export function errorRecord(e: unknown, r: { id: string; method: string; route: string | null; path: string; status: number }): ServerError { | |
| 42 | + const err = e instanceof Error ? e : null; | |
| 43 | + return { | |
| 44 | + id: r.id, | |
| 45 | + at: new Date().toISOString(), | |
| 46 | + method: r.method, | |
| 47 | + route: r.route, | |
| 48 | + path: pathToKeep(r.path, r.route), | |
| 49 | + status: r.status, | |
| 50 | + message: redact(err ? `${err.name}: ${err.message}` : String(e)).slice(0, 2000), | |
| 51 | + stack: err?.stack ? redact(err.stack).slice(0, 8000) : null, | |
| 52 | + }; | |
| 53 | +} | |
| 54 | + | |
| 55 | +/** Saves an error and drops all but the newest KEEP_ERRORS. Never throws: the database may be what failed. */ | |
| 56 | +export async function recordServerError(env: Env, rec: ServerError): Promise<void> { | |
| 57 | + try { | |
| 58 | + await env.DB.prepare("INSERT OR REPLACE INTO server_errors (id, at, method, route, path, status, message, stack) VALUES (?, ?, ?, ?, ?, ?, ?, ?)") | |
| 59 | + .bind(rec.id, rec.at, rec.method, rec.route, rec.path, rec.status, rec.message, rec.stack) | |
| 60 | + .run(); | |
| 61 | + await env.DB.prepare("DELETE FROM server_errors WHERE at < (SELECT at FROM server_errors ORDER BY at DESC LIMIT 1 OFFSET ?)").bind(KEEP_ERRORS - 1).run(); | |
| 62 | + } catch (x) { | |
| 63 | + console.error("could not record a server error", x); | |
| 64 | + } | |
| 65 | +} | |
| 66 | + | |
| 67 | +/** The newest errors first, or the one with request id `id`. */ | |
| 68 | +export async function serverErrors(env: Env, o: { limit?: number; id?: string } = {}): Promise<ServerError[]> { | |
| 69 | + if (o.id) return (await env.DB.prepare("SELECT * FROM server_errors WHERE id = ?").bind(o.id).all<ServerError>()).results; | |
| 70 | + const limit = Math.min(Math.max(o.limit ?? 100, 1), KEEP_ERRORS); | |
| 71 | + return (await env.DB.prepare("SELECT * FROM server_errors ORDER BY at DESC LIMIT ?").bind(limit).all<ServerError>()).results; | |
| 72 | +} |
modifiedapps/diffinery/src/web/account.tsx+86−2View file
| @@ -20,6 +20,7 @@ | ||
| 20 | 20 | import { MAX_AVATAR_BYTES } from "../lib/image.js"; |
| 21 | 21 | import { gatherToday } from "../agentic/today.js"; |
| 22 | 22 | import { TodayView } from "../agentic/web.js"; |
| 23 | +import { serverErrors } from "../lib/server-errors.js"; | |
| 23 | 24 | import { ComputePausedBanner, KillSwitchesPage } from "./kill-switches.js"; |
| 24 | 25 | import { changePause, changeSuspension, cleanReason, type Stopped } from "../lib/killswitch.js"; |
| 25 | 26 | |
| @@ -643,9 +644,9 @@ | ||
| 643 | 644 | |
| 644 | 645 | // ---------------------------------------------------------------- /settings |
| 645 | 646 | |
| 646 | -export type SettingsTab = "profile" | "appearance" | "security" | "tokens" | "agents" | "model-access" | "orgs" | "invitations" | "notifications" | "linked" | "admin" | "email" | "audit" | "kill-switches"; | |
| 647 | +export type SettingsTab = "profile" | "appearance" | "security" | "tokens" | "agents" | "model-access" | "orgs" | "invitations" | "notifications" | "linked" | "admin" | "email" | "audit" | "kill-switches" | "errors"; | |
| 647 | 648 | /** The site administration pages: they show when hosted compute is paused. */ |
| 648 | -export const ADMIN_TABS: readonly SettingsTab[] = ["admin", "email", "audit", "kill-switches"]; | |
| 649 | +export const ADMIN_TABS: readonly SettingsTab[] = ["admin", "email", "audit", "kill-switches", "errors"]; | |
| 649 | 650 | |
| 650 | 651 | export const SettingsNav: FC<{ me: Account; tab: SettingsTab }> = ({ me, tab }) => ( |
| 651 | 652 | <aside class="side vt-side"> |
| @@ -704,6 +705,9 @@ | ||
| 704 | 705 | <a href="/settings/audit" class={tab === "audit" ? "on" : ""}> |
| 705 | 706 | Audit log |
| 706 | 707 | </a> |
| 708 | + <a href="/settings/admin/errors" class={tab === "errors" ? "on" : ""}> | |
| 709 | + Server errors | |
| 710 | + </a> | |
| 707 | 711 | </> |
| 708 | 712 | ) : null} |
| 709 | 713 | </div> |
| @@ -1710,6 +1714,86 @@ | ||
| 1710 | 1714 | </>, |
| 1711 | 1715 | ); |
| 1712 | 1716 | }); |
| 1717 | + | |
| 1718 | + // Unexpected server errors (lib/server-errors.ts); ?id= finds the one a 5xx answer's request ID names. | |
| 1719 | + web.get("/settings/admin/errors", async (c) => { | |
| 1720 | + const me = requireSiteAdmin(c); | |
| 1721 | + const id = c.req.query("id")?.trim() || undefined; | |
| 1722 | + const rows = await serverErrors(c.env, { id, limit: 200 }); | |
| 1723 | + return page( | |
| 1724 | + c, | |
| 1725 | + me, | |
| 1726 | + "errors", | |
| 1727 | + "Server errors", | |
| 1728 | + <> | |
| 1729 | + <h1>Server errors</h1> | |
| 1730 | + <p class="muted"> | |
| 1731 | + {id ? ( | |
| 1732 | + <> | |
| 1733 | + The error with request ID <span class="mono">{id}</span>. | |
| 1734 | + </> | |
| 1735 | + ) : ( | |
| 1736 | + <> | |
| 1737 | + The 200 most recent answers with a 5xx status. Each such answer carries its request ID (<span class="mono">request_id</span> and the <span class="mono">x-request-id</span> header). | |
| 1738 | + </> | |
| 1739 | + )}{" "} | |
| 1740 | + No request bodies, query strings or headers are kept. | |
| 1741 | + </p> | |
| 1742 | + <form method="get" action="/settings/admin/errors" class="row" style="gap:8px;margin-bottom:12px"> | |
| 1743 | + <input type="search" name="id" placeholder="Request ID" value={id ?? ""} class="mono" style="flex:1;max-width:360px" /> | |
| 1744 | + <button class="btn" type="submit"> | |
| 1745 | + Find | |
| 1746 | + </button> | |
| 1747 | + {id ? ( | |
| 1748 | + <a class="btn" href="/settings/admin/errors"> | |
| 1749 | + Show all | |
| 1750 | + </a> | |
| 1751 | + ) : null} | |
| 1752 | + </form> | |
| 1753 | + <div class="box" style="overflow:auto"> | |
| 1754 | + <table class="tbl"> | |
| 1755 | + <thead> | |
| 1756 | + <tr> | |
| 1757 | + <th>When</th> | |
| 1758 | + <th>Request</th> | |
| 1759 | + <th>Status</th> | |
| 1760 | + <th>Error</th> | |
| 1761 | + </tr> | |
| 1762 | + </thead> | |
| 1763 | + <tbody> | |
| 1764 | + {rows.map((r) => ( | |
| 1765 | + <tr> | |
| 1766 | + <td class="small nowrap"> | |
| 1767 | + <Time at={r.at} /> | |
| 1768 | + </td> | |
| 1769 | + <td class="small mono"> | |
| 1770 | + <div> | |
| 1771 | + {r.method} {r.route ?? r.path} | |
| 1772 | + </div> | |
| 1773 | + <div class="muted">{r.id}</div> | |
| 1774 | + </td> | |
| 1775 | + <td> | |
| 1776 | + <span class="badge">{r.status}</span> | |
| 1777 | + </td> | |
| 1778 | + <td class="small mono" style="max-width:520px"> | |
| 1779 | + {r.stack ? ( | |
| 1780 | + <details> | |
| 1781 | + <summary style="overflow:hidden;text-overflow:ellipsis;white-space:nowrap">{r.message}</summary> | |
| 1782 | + <pre class="muted" style="white-space:pre-wrap">{r.stack}</pre> | |
| 1783 | + </details> | |
| 1784 | + ) : ( | |
| 1785 | + r.message | |
| 1786 | + )} | |
| 1787 | + </td> | |
| 1788 | + </tr> | |
| 1789 | + ))} | |
| 1790 | + </tbody> | |
| 1791 | + </table> | |
| 1792 | + {rows.length === 0 ? <div class="empty">{id ? "No error with that request ID; only the newest 500 are kept." : "No server errors."}</div> : null} | |
| 1793 | + </div> | |
| 1794 | + </>, | |
| 1795 | + ); | |
| 1796 | + }); | |
| 1713 | 1797 | } |
| 1714 | 1798 | |
| 1715 | 1799 | // ---------------------------------------------------------------- /:owner profiles |
modifiedapps/diffinery/src/web/ui.tsx+8−1View file
| @@ -8,7 +8,7 @@ | ||
| 8 | 8 | import type { AppContext, AppVars } from "../auth/principal.js"; |
| 9 | 9 | import { HttpError, randomToken } from "../lib/util.js"; |
| 10 | 10 | import { atLeast, type Caller } from "../repo/types.js"; |
| 11 | -import { RepoMoved, type repoStub } from "../http/common.js"; | |
| 11 | +import { noteServerError, RepoMoved, type repoStub } from "../http/common.js"; | |
| 12 | 12 | import { CSS, ICONS, JS, type IconName } from "./assets.js"; |
| 13 | 13 | import { CSRF_FIELD, csrfToken, verifyCsrf, type FormData } from "./csrf.js"; |
| 14 | 14 | import { fenceLangs, renderMarkdown, type MarkdownBase } from "./markdown.js"; |
| @@ -541,6 +541,8 @@ | ||
| 541 | 541 | if (confirm) return c.redirect(confirm, 303); |
| 542 | 542 | const s = statusOf(e); |
| 543 | 543 | if (s.status >= 500) console.error(e); |
| 544 | + // Site administrators find the error by this id (Settings -> Server errors). | |
| 545 | + const requestId = s.status >= 500 ? noteServerError(c, e, s.status) : null; | |
| 544 | 546 | // Signed-out visitors who open a page go straight to sign-in, and come back afterwards. |
| 545 | 547 | if (s.status === 401 && c.req.method === "GET" && c.get("principal")?.type === "anonymous" && !c.req.path.startsWith("/auth/")) { |
| 546 | 548 | const url = new URL(c.req.url); |
| @@ -569,6 +571,11 @@ | ||
| 569 | 571 | ) : ( |
| 570 | 572 | <p class="muted">{s.status >= 500 ? "An unexpected error occurred. Please try again." : s.message}</p> |
| 571 | 573 | )} |
| 574 | + {requestId ? ( | |
| 575 | + <p class="small muted"> | |
| 576 | + Request ID <span class="mono">{requestId}</span> | |
| 577 | + </p> | |
| 578 | + ) : null} | |
| 572 | 579 | {s.status === 403 && p?.type === "account" && p.account.status === "pending" ? ( |
| 573 | 580 | <p> |
| 574 | 581 | <a class="btn primary" href="/"> |
addedapps/diffinery/test/unit/server-errors.test.ts+68−0View file
| @@ -1,0 +1,68 @@ | ||
| 1 | +import { Hono } from "hono"; | |
| 2 | +import { describe, expect, it } from "vitest"; | |
| 3 | +import type { Env } from "../../src/env.js"; | |
| 4 | +import { errorResponse } from "../../src/http/common.js"; | |
| 5 | +import { HttpError } from "../../src/lib/util.js"; | |
| 6 | +import { errorRecord, KEEP_ERRORS, pathToKeep, recordServerError, redact, serverErrors } from "../../src/lib/server-errors.js"; | |
| 7 | +import { testD1 } from "./stubs/d1.js"; | |
| 8 | + | |
| 9 | +const settle = () => new Promise((r) => setTimeout(r, 0)); | |
| 10 | + | |
| 11 | +describe("server errors", () => { | |
| 12 | + it("mask tokens and keep secret-carrying paths as their route", () => { | |
| 13 | + expect(redact("push to https://x-token:dfy_abcdefghijklmnop@diffinery.com/a/b.git failed")).toBe("push to https://…@diffinery.com/a/b.git failed"); | |
| 14 | + expect(redact("token dfy_abcdefghijklmnop and ghs_0123456789abcdef")).toBe("token dfy_… and ghs_…"); | |
| 15 | + expect(redact("Authorization: Bearer abc.def.ghijklmnop")).toBe("Authorization: Bearer …"); | |
| 16 | + expect(pathToKeep("/signin/s3cr3t", "/signin/:token")).toBe("/signin/:token"); | |
| 17 | + expect(pathToKeep("/api/v3/repos/a/b/actions/secrets/X", "/api/v3/repos/:owner/:repo/actions/secrets/:name")).toBe("/api/v3/repos/a/b/actions/secrets/X"); | |
| 18 | + }); | |
| 19 | + | |
| 20 | + it("a 5xx answer carries a request id that finds its record; other answers aren't recorded", async () => { | |
| 21 | + const env = { DB: testD1() } as unknown as Env; | |
| 22 | + const app = new Hono<{ Bindings: Env }>(); | |
| 23 | + app.onError((e, c) => errorResponse(e, c as never)); | |
| 24 | + app.put("/api/v3/repos/:owner/:repo/actions/secrets/:name", () => { | |
| 25 | + throw new Error("D1_ERROR: storage operation exceeded timeout, using dfy_abcdefghijklmnop"); | |
| 26 | + }); | |
| 27 | + app.get("/signin/:token", () => { | |
| 28 | + throw new Error("boom"); | |
| 29 | + }); | |
| 30 | + app.get("/missing", () => { | |
| 31 | + throw new HttpError(404, "Not Found"); | |
| 32 | + }); | |
| 33 | + | |
| 34 | + const res = await app.request("/api/v3/repos/acme/app/actions/secrets/CLOUDFLARE_API_TOKEN?x=1", { method: "PUT", headers: { "cf-ray": "8d2f3a1b4c5d6e7f-FRA" }, body: "{\"encrypted_value\":\"secret\"}" }, env); | |
| 35 | + expect(res.status).toBe(500); | |
| 36 | + const body = (await res.json()) as { message: string; request_id: string }; | |
| 37 | + expect(body).toEqual({ message: "internal error", request_id: "8d2f3a1b4c5d6e7f-FRA" }); | |
| 38 | + expect(res.headers.get("x-request-id")).toBe("8d2f3a1b4c5d6e7f-FRA"); | |
| 39 | + expect((await app.request("/signin/s3cr3t", {}, env)).status).toBe(500); | |
| 40 | + expect((await app.request("/missing", {}, env)).status).toBe(404); | |
| 41 | + await settle(); | |
| 42 | + | |
| 43 | + const [e] = await serverErrors(env, { id: body.request_id }); | |
| 44 | + expect(e).toMatchObject({ | |
| 45 | + method: "PUT", | |
| 46 | + route: "/api/v3/repos/:owner/:repo/actions/secrets/:name", | |
| 47 | + path: "/api/v3/repos/acme/app/actions/secrets/CLOUDFLARE_API_TOKEN", | |
| 48 | + status: 500, | |
| 49 | + message: "Error: D1_ERROR: storage operation exceeded timeout, using dfy_…", | |
| 50 | + }); | |
| 51 | + expect(e!.stack).toContain("D1_ERROR"); | |
| 52 | + expect(JSON.stringify(await serverErrors(env))).not.toMatch(/s3cr3t|encrypted_value|x=1|abcdefghijklmnop/); | |
| 53 | + expect((await serverErrors(env)).map((r) => r.path)).toEqual(expect.arrayContaining(["/signin/:token"])); | |
| 54 | + expect(await serverErrors(env)).toHaveLength(2); | |
| 55 | + }); | |
| 56 | + | |
| 57 | + it("keep only the newest records", async () => { | |
| 58 | + const env = { DB: testD1() } as unknown as Env; | |
| 59 | + for (let i = 0; i < KEEP_ERRORS + 5; i++) { | |
| 60 | + const r = errorRecord(new Error(`e${i}`), { id: `r${i}`, method: "GET", route: "/x", path: "/x", status: 500 }); | |
| 61 | + await recordServerError(env, { ...r, at: new Date(Date.UTC(2026, 9, 14, 0, 0, i)).toISOString() }); | |
| 62 | + } | |
| 63 | + const all = await serverErrors(env, { limit: 1000 }); | |
| 64 | + expect(all).toHaveLength(KEEP_ERRORS); | |
| 65 | + expect(all[0]!.id).toBe(`r${KEEP_ERRORS + 4}`); | |
| 66 | + expect(await serverErrors(env, { id: "r4" })).toEqual([]); | |
| 67 | + }); | |
| 68 | +}); |