Security e2e wait-timer test takes seconds; run ends settle their task first; Terminals button on the Project page (#129)
* Environment wait timers: a test-only minute length, so the security e2e waits seconds, not minutes The reviewers test waited out a real one-minute wait timer and then the run's Workflow, which took 366 s on CI and pushed the security shard past its 600 s limit. WAIT_TIMER_MINUTE_MS (end-to-end tests only, default 60000) sets how long a wait-timer minute lasts; the security e2e uses 10 s. The test now takes about 36 s and the whole file about 3.5 minutes locally. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014VDQqT8oL18uLJ9fpsNxW2 * A Terminals button on the Project page Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014VDQqT8oL18uLJ9fpsNxW2 * WAIT_TIMER_MINUTE_MS counts only on a Worker on this machine; a deployed one ignores it The setting shortens a deployment protection rule, so like DEV_AUTH it takes effect only when PUBLIC_URL is a loopback address and Cloudflare Access is off. Unit test covers the production case. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014VDQqT8oL18uLJ9fpsNxW2 * Tasks: a run's end settles its task before the sandbox closes Security's patch (projectdo-endrun-race): endRun marked the run ended, then awaited the agent's kill and the sandbox's close before changing the task, so a read in between saw the run done while its task still ran (the plan-first e2e flaked on it). The task's state and events now change with the run's, and Today is indexed before the close; a waiting message starts its run after the close. On top of the patch: a stop made while the sandbox closed keeps its state instead of being settled over. Unit test (with a cloudflare:workers stub for vitest) reads between the two writes. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014VDQqT8oL18uLJ9fpsNxW2 --------- Co-authored-by: Claude <noreply@anthropic.com>
Canberk Demirci authored · GitHub committed signed1 parent 3a39201commit 7a4c6b290b3199d36fbae8fadaaf0a829ae845f9Browse files
12 files changed+208−36
Files (12)
- apps/diffinery/docs/agents/terminal.md+1−1
- apps/diffinery/docs/self-hosting/configuration.md+1−0
- apps/diffinery/src/agentic/ProjectDO.ts+38−21
- apps/diffinery/src/agentic/project-web.tsx+6−0
- apps/diffinery/src/auth/dev.ts+8−0
- apps/diffinery/src/env.ts+2−0
- apps/diffinery/src/repo/actions.ts+14−2
- apps/diffinery/test/e2e/security.test.mjs+7−11
- apps/diffinery/test/unit/project-endrun.test.ts+95−0
- apps/diffinery/test/unit/stubs/cloudflare-workers.ts+7−0
- apps/diffinery/test/unit/wait-timer.test.ts+23−0
- apps/diffinery/vitest.config.ts+6−1
modifiedapps/diffinery/docs/agents/terminal.md+1−1View file
| @@ -17,7 +17,7 @@ | ||
| 17 | 17 | |
| 18 | 18 | ## Open a terminal |
| 19 | 19 | |
| 20 | -1. Open the repository's **Agent tasks** page (`/<owner>/<repo>/tasks`) and click **Terminals**, or go to `/<owner>/<repo>/terminals`. | |
| 20 | +1. Open the repository's **Project** tab and click **Terminals** (the **Agent tasks** page has the same button), or go to `/<owner>/<repo>/terminals`. | |
| 21 | 21 | 2. Pick the **agent** and the **branch**. |
| 22 | 22 | 3. The first time, tick the notice: you use Claude Code and Codex with your own Anthropic or OpenAI account, under that provider's terms, and usage is billed to your account. Diffinery records when you accepted it and which version of the notice it was. When the notice changes, you're asked again. |
| 23 | 23 | 4. Click **Open a terminal**. |
modifiedapps/diffinery/docs/self-hosting/configuration.md+1−0View file
| @@ -36,6 +36,7 @@ | ||
| 36 | 36 | | `GITHUB_OAUTH_CLIENT_ID` | Not set | No | The client ID of a GitHub OAuth app, for linking GitHub accounts with a GitHub sign-in instead of a gist. Needs `GITHUB_OAUTH_CLIENT_SECRET` too. See [GitHub imports](#github-imports). | |
| 37 | 37 | | `IMPORT_GIT_AGENT` | Not set in production; `external` in the `dev` environment | Never in production | With `external`, an import's Git transfer doesn't start a container: it waits for an import agent started by hand (see [The dev environment](#the-dev-environment)). | |
| 38 | 38 | | `SCHEDULE_TICK_MINUTES` | Not set | No | Minutes between the workflow scheduler's checks, a whole number from 1 to 60. Default `5`, GitHub's shortest schedule interval. Anything else counts as not set. The end-to-end tests use `1`. | |
| 39 | +| `WAIT_TIMER_MINUTE_MS` | Not set | Never in production | Local development and the end-to-end tests only: how long one minute of a deployment environment's wait timer lasts, in milliseconds, from 1 to `60000` (the default). Anything else counts as not set. It counts only when `PUBLIC_URL` is on this machine (`localhost`, `127.x` or `[::1]`) and `ACCESS_AUD` is empty, as for `DEV_AUTH`; a deployed Worker ignores it. The end-to-end tests use `10000` so a wait timer passes in seconds. | | |
| 39 | 40 | | `TERMINALS` | Not set | No | `off` switches [terminals](/docs/agents/terminal) off: nobody can open one, and open ones end at their next check, within a minute. Anything else, or not set, leaves them on. | |
| 40 | 41 | |
| 41 | 42 | When either `ACCESS_TEAM_DOMAIN` or `ACCESS_AUD` is empty, email sign-in at `/auth/access` is off: the page answers "Account recovery by email is not set up on this instance. Ask an administrator for a sign-in link.", and the sign-in page doesn't offer **Recover your account by email**. Passkeys, invitations and admin sign-in links keep working, but the first administrator can't sign in, because email sign-in is how an account in `BOOTSTRAP_ADMINS` is created. |
modifiedapps/diffinery/src/agentic/ProjectDO.ts+38−21View file
| @@ -1668,7 +1668,10 @@ | ||
| 1668 | 1668 | this.arm(0); |
| 1669 | 1669 | } |
| 1670 | 1670 | |
| 1671 | - /** End a run: revoke its model token, kill the agent, close the sandbox (which revokes its git credential), then start a queued message. */ | |
| 1671 | + /** | |
| 1672 | + * End a run: revoke its model token, settle the task's state, kill the agent, close the sandbox | |
| 1673 | + * (which revokes its git credential), then start a queued message. | |
| 1674 | + */ | |
| 1672 | 1675 | private async endRun(r: RunRow, step: "done" | "failed" | "stopped", error: string | null): Promise<void> { |
| 1673 | 1676 | const at = now(); |
| 1674 | 1677 | const current = this.run(r.task, r.seq); |
| @@ -1676,16 +1679,9 @@ | ||
| 1676 | 1679 | r = current; |
| 1677 | 1680 | this.sql.exec("UPDATE runs SET step = ?, error = ?, token_hash = NULL, ended_at = ? WHERE task = ? AND seq = ?", step, error, at, r.task, r.seq); |
| 1678 | 1681 | if (error) this.note(r.task, r.seq, error); |
| 1679 | - if (r.sandbox_id) { | |
| 1680 | - if (r.proc) await this.sandbox(r, "/proc/kill", { id: r.proc, signal: "SIGKILL" }).catch(() => {}); | |
| 1681 | - await this.env.REPO.get(this.env.REPO.idFromName(this.mustTask(r.task).repo_id)) | |
| 1682 | - .closeTaskSandbox(r.sandbox_id) | |
| 1683 | - .catch((e) => console.error("closing a task sandbox failed", e)); | |
| 1684 | - } | |
| 1682 | + // The task's state and posts change with the run's, before anything is awaited: a reader never | |
| 1683 | + // sees the run ended while its task still runs, and a message sent meanwhile isn't overwritten. | |
| 1685 | 1684 | const t = this.mustTask(r.task); |
| 1686 | - if (t.kind === "review" && step !== "done" && r.head_sha) { | |
| 1687 | - await this.reviewCheck(t, r.head_sha, "completed", step === "stopped" ? "cancelled" : "failure", step === "stopped" ? "The review was stopped" : "The review didn't finish", error ?? ""); | |
| 1688 | - } | |
| 1689 | 1685 | if (t.kind === "coordinator" && step !== "done") { |
| 1690 | 1686 | const posts = (JSON.parse(r.message ?? "[]") as number[]).map((p) => `post #${p}`).join(", "); |
| 1691 | 1687 | // Before `finish`, nothing has been applied; `coordinate` says so itself when it rejects the actions. |
| @@ -1697,6 +1693,15 @@ | ||
| 1697 | 1693 | this.sql.exec("UPDATE tasks SET queued = NULL WHERE number = ?", r.task); |
| 1698 | 1694 | t.queued = null; |
| 1699 | 1695 | } |
| 1696 | + const settle = () => { | |
| 1697 | + this.sql.exec("UPDATE tasks SET state = ?, error = ?, updated_at = ? WHERE number = ?", step === "done" ? "ready" : step, error, at, r.task); | |
| 1698 | + if (step === "failed") this.event(r.task, `Task #${r.task} failed: ${error ?? "the run failed"}`); | |
| 1699 | + else if (step === "stopped") this.event(r.task, `Task #${r.task} stopped: ${error ?? "stopped"}`); | |
| 1700 | + else if (r.kind === "plan") this.event(r.task, `Task #${r.task}'s plan needs approval.`); | |
| 1701 | + else if (!this.task(r.task)!.pr_number) this.event(r.task, `Task #${r.task} finished with no changes to review.`); | |
| 1702 | + }; | |
| 1703 | + // A queued message starts a new run once this one's sandbox is closed (below); the task stays running. | |
| 1704 | + const queued = !(r.attempt && (this.activeRun(r.task) || t.pick_state === "running")) && !!t.queued && step !== "stopped"; | |
| 1700 | 1705 | if (r.attempt && this.activeRun(r.task)) { |
| 1701 | 1706 | // Other attempts are still running: the task's state waits for the last of them. |
| 1702 | 1707 | } else if (r.attempt && t.pick_state === "running") { |
| @@ -1717,20 +1722,32 @@ | ||
| 1717 | 1722 | this.sql.exec("UPDATE tasks SET state = 'ready', pick_state = NULL, error = NULL, updated_at = ? WHERE number = ?", at, r.task); |
| 1718 | 1723 | this.event(r.task, `Task #${r.task}'s ${all} attempts finished with no changes to review.`); |
| 1719 | 1724 | } |
| 1720 | - } else if (t.queued && step !== "stopped") { | |
| 1721 | - this.sql.exec("UPDATE tasks SET queued = NULL WHERE number = ?", r.task); | |
| 1722 | - if (t.kind === "review") await this.restartReview(t, t.queued === "automatic" ? null : t.queued, JSON.parse(r.providers) as Provider[]); | |
| 1723 | - else this.startRun(r.task, t.kind === "coordinator" ? "coordinate" : this.task(r.task)!.plan_state === "planning" || this.task(r.task)!.plan_state === "waiting" ? "plan" : "message", t.queued, JSON.parse(r.providers) as Provider[]); | |
| 1724 | - } else { | |
| 1725 | - this.sql.exec("UPDATE tasks SET state = ?, error = ?, updated_at = ? WHERE number = ?", step === "done" ? "ready" : step, error, at, r.task); | |
| 1726 | - if (step === "failed") this.event(r.task, `Task #${r.task} failed: ${error ?? "the run failed"}`); | |
| 1727 | - else if (step === "stopped") this.event(r.task, `Task #${r.task} stopped: ${error ?? "stopped"}`); | |
| 1728 | - else if (r.kind === "plan") this.event(r.task, `Task #${r.task}'s plan needs approval.`); | |
| 1729 | - else if (!this.task(r.task)!.pr_number) this.event(r.task, `Task #${r.task} finished with no changes to review.`); | |
| 1725 | + } else if (!queued) settle(); | |
| 1726 | + // Today (D1) shows the settled state now too, not once the sandbox has closed. | |
| 1727 | + await this.index(r.task); | |
| 1728 | + if (r.sandbox_id) { | |
| 1729 | + if (r.proc) await this.sandbox(r, "/proc/kill", { id: r.proc, signal: "SIGKILL" }).catch(() => {}); | |
| 1730 | + await this.env.REPO.get(this.env.REPO.idFromName(t.repo_id)) | |
| 1731 | + .closeTaskSandbox(r.sandbox_id) | |
| 1732 | + .catch((e) => console.error("closing a task sandbox failed", e)); | |
| 1730 | 1733 | } |
| 1734 | + if (t.kind === "review" && step !== "done" && r.head_sha) { | |
| 1735 | + await this.reviewCheck(t, r.head_sha, "completed", step === "stopped" ? "cancelled" : "failure", step === "stopped" ? "The review was stopped" : "The review didn't finish", error ?? ""); | |
| 1736 | + } | |
| 1737 | + if (queued && !this.activeRun(r.task)) { | |
| 1738 | + // Read again: a stop while the sandbox closed dropped the queued message, and settled the task itself. | |
| 1739 | + const fresh = this.mustTask(r.task); | |
| 1740 | + if (!fresh.queued) { | |
| 1741 | + if (fresh.state === "running") settle(); | |
| 1742 | + } else { | |
| 1743 | + this.sql.exec("UPDATE tasks SET queued = NULL WHERE number = ?", r.task); | |
| 1744 | + if (fresh.kind === "review") await this.restartReview(fresh, fresh.queued === "automatic" ? null : fresh.queued, JSON.parse(r.providers) as Provider[]); | |
| 1745 | + else this.startRun(r.task, fresh.kind === "coordinator" ? "coordinate" : fresh.plan_state === "planning" || fresh.plan_state === "waiting" ? "plan" : "message", fresh.queued, JSON.parse(r.providers) as Provider[]); | |
| 1746 | + } | |
| 1747 | + } | |
| 1731 | 1748 | // Its slot is free: the next queued task starts. |
| 1732 | 1749 | if (RUNNING.includes(r.step)) this.promote(); |
| 1733 | - await this.index(r.task); | |
| 1750 | + if (queued) await this.index(r.task); | |
| 1734 | 1751 | } |
| 1735 | 1752 | |
| 1736 | 1753 | /** Start a queued review on the pull request's current head, if it is still open. */ |
modifiedapps/diffinery/src/agentic/project-web.tsx+6−0View file
| @@ -15,6 +15,7 @@ | ||
| 15 | 15 | import { handlePost, loadBoard, postToProject, saveCoordinator, saveMemory, saveProjectSettings, type Handled } from "./projects.js"; |
| 16 | 16 | import { parseRuntimeChoice, runtimeChoices, runtimeLabel } from "./runtimes.js"; |
| 17 | 17 | import { drivableAgents, projectStub, requireTaskReader } from "./tasks.js"; |
| 18 | +import { terminalsEnabled } from "./terminals.js"; | |
| 18 | 19 | import { OpenRouterModel, TaskBadge } from "./tasks-web.js"; |
| 19 | 20 | |
| 20 | 21 | /** |
| @@ -187,6 +188,11 @@ | ||
| 187 | 188 | <a class="btn" href={`${base}/tasks`}> |
| 188 | 189 | All tasks |
| 189 | 190 | </a> |
| 191 | + {writable && terminalsEnabled(c.env) ? ( | |
| 192 | + <a class="btn" href={`${base}/terminals`}> | |
| 193 | + Terminals | |
| 194 | + </a> | |
| 195 | + ) : null} | |
| 190 | 196 | {writable ? ( |
| 191 | 197 | <a class="btn primary" href={`${base}/tasks/new`}> |
| 192 | 198 | New task |
modifiedapps/diffinery/src/auth/dev.ts+8−0View file
| @@ -20,3 +20,11 @@ | ||
| 20 | 20 | if (!env.DEV_AUTH || env.ACCESS_AUD || !loopback(env.PUBLIC_URL)) return false; |
| 21 | 21 | return requestUrl === undefined || loopback(requestUrl); |
| 22 | 22 | } |
| 23 | + | |
| 24 | +/** | |
| 25 | + * Whether this Worker runs on this machine only (wrangler dev, the end-to-end tests): PUBLIC_URL on a loopback | |
| 26 | + * address and Cloudflare Access off. Test-only settings that loosen a control count only then. | |
| 27 | + */ | |
| 28 | +export function localOnly(env: Pick<Env, "ACCESS_AUD" | "PUBLIC_URL">): boolean { | |
| 29 | + return !env.ACCESS_AUD && loopback(env.PUBLIC_URL); | |
| 30 | +} |
modifiedapps/diffinery/src/env.ts+2−0View file
| @@ -15,6 +15,8 @@ | ||
| 15 | 15 | SCHEDULER: DurableObjectNamespace<SchedulerDO>; |
| 16 | 16 | /** Minutes between scheduler ticks, 1 to 60 (default 5, GitHub's shortest schedule interval). */ |
| 17 | 17 | SCHEDULE_TICK_MINUTES?: string; |
| 18 | + /** End-to-end tests only: how long one minute of an environment's wait timer lasts, in ms (default 60000). */ | |
| 19 | + WAIT_TIMER_MINUTE_MS?: string; | |
| 18 | 20 | RUNNER: DurableObjectNamespace<RunnerContainer>; |
| 19 | 21 | SANDBOX: DurableObjectNamespace<SandboxContainer>; |
| 20 | 22 | /** One per repository: its agent tasks (agentic/ProjectDO.ts). */ |
modifiedapps/diffinery/src/repo/actions.ts+14−2View file
| @@ -22,6 +22,7 @@ | ||
| 22 | 22 | import { signRuntimeToken } from "../lib/runtime-auth.js"; |
| 23 | 23 | import { ghRepo, numericId } from "../http/gh.js"; |
| 24 | 24 | import { HttpError, newId, now, randomToken, sha256 } from "../lib/util.js"; |
| 25 | +import { localOnly } from "../auth/dev.js"; | |
| 25 | 26 | import { endCompute, mayStart, startCompute } from "../lib/quotas.js"; |
| 26 | 27 | import { blockedBy, killSwitches, type Work } from "../lib/killswitch.js"; |
| 27 | 28 | import type { Env } from "../env.js"; |
| @@ -1096,7 +1097,7 @@ | ||
| 1096 | 1097 | this.repo.broadcast(`run:${run.id}`, { action: "job", jobId: job.id }); |
| 1097 | 1098 | this.repo.broadcast("runs", { action: "waiting", runId: run.id }); |
| 1098 | 1099 | } |
| 1099 | - const timerEnds = Date.parse(since!) + rules.waitTimer * 60_000; | |
| 1100 | + const timerEnds = Date.parse(since!) + rules.waitTimer * timerMinute(this.env); | |
| 1100 | 1101 | if ((rules.reviewers.length && decision !== "approved") || Date.now() < timerEnds) return false; |
| 1101 | 1102 | this.release({ ...job, status: "approval" }); |
| 1102 | 1103 | return true; |
| @@ -1164,7 +1165,7 @@ | ||
| 1164 | 1165 | } |
| 1165 | 1166 | const decided = this.repo.deploys.decision(job.run_id, job.environment!) === "approved"; |
| 1166 | 1167 | if (rules.reviewers.length && !decided) return { status: "approval", wait: "review", until: new Date(deadline).toISOString() }; |
| 1167 | - return { status: "approval", wait: "timer", until: new Date(since + rules.waitTimer * 60_000).toISOString() }; | |
| 1168 | + return { status: "approval", wait: "timer", until: new Date(since + rules.waitTimer * timerMinute(this.env)).toISOString() }; | |
| 1168 | 1169 | } |
| 1169 | 1170 | |
| 1170 | 1171 | /** The environments a run's jobs wait on, with who may approve them (GitHub's pending deployments). */ |
| @@ -2166,3 +2167,14 @@ | ||
| 2166 | 2167 | return { runs, jobs, sandboxes }; |
| 2167 | 2168 | } |
| 2168 | 2169 | } |
| 2170 | + | |
| 2171 | +/** | |
| 2172 | + * One minute of a wait timer: a real minute, except in the end-to-end tests (WAIT_TIMER_MINUTE_MS). The setting | |
| 2173 | + * shortens a protection rule, so it counts only on a Worker that runs on this machine (auth/dev.ts): a deployed | |
| 2174 | + * one ignores it. | |
| 2175 | + */ | |
| 2176 | +export function timerMinute(env: Pick<Env, "WAIT_TIMER_MINUTE_MS" | "PUBLIC_URL" | "ACCESS_AUD">): number { | |
| 2177 | + if (!env.WAIT_TIMER_MINUTE_MS || !localOnly(env)) return 60_000; | |
| 2178 | + const ms = Number(env.WAIT_TIMER_MINUTE_MS); | |
| 2179 | + return Number.isInteger(ms) && ms > 0 && ms <= 60_000 ? ms : 60_000; | |
| 2180 | +} |
modifiedapps/diffinery/test/e2e/security.test.mjs+7−11View file
| @@ -44,6 +44,8 @@ | ||
| 44 | 44 | const T = { sully: tok("sully"), olive: tok("olive"), bot: tok("bot"), rita: tok("rita"), dora: tok("dora"), orphan: tok("orphan"), reader: tok("reader"), ritaAdmin: tok("ritaAdmin"), milo: tok("milo"), sam: tok("sam"), helper: tok("helper") }; |
| 45 | 45 | const S = { penny: `penny-session-${randomBytes(12).toString("hex")}`, dora: `dora-session-${randomBytes(12).toString("hex")}`, olive: `olive-session-${randomBytes(12).toString("hex")}`, sam: `sam-session-${randomBytes(12).toString("hex")}`, oliveOld: `olive-old-${randomBytes(12).toString("hex")}`, oliveStale: `olive-stale-${randomBytes(12).toString("hex")}`, oliveSpare: `olive-spare-${randomBytes(12).toString("hex")}` }; |
| 46 | 46 | const OWNER = "olive"; |
| 47 | +/** One minute of an environment's wait timer, in this Worker (WAIT_TIMER_MINUTE_MS). */ | |
| 48 | +const TIMER_MINUTE_MS = 10_000; | |
| 47 | 49 | |
| 48 | 50 | const freePort = () => |
| 49 | 51 | new Promise((res) => { |
| @@ -181,7 +183,7 @@ | ||
| 181 | 183 | ); |
| 182 | 184 | dev = spawn( |
| 183 | 185 | "pnpm", |
| 184 | - ["exec", "wrangler", "dev", "-e", "dev", "--ip", "127.0.0.1", "--port", String(port), "--inspector-port", String(await freePort()), "--persist-to", persist, "--var", `PUBLIC_URL:${base}`, "--var", "DEV_AUTH:", "--var", `MASTER_KEY:${randomBytes(32).toString("base64")}`, "--var", `OIDC_PRIVATE_KEY:${JSON.stringify(generateKeyPairSync("ec", { namedCurve: "P-256" }).privateKey.export({ format: "jwk" }))}`, "--show-interactive-dev-session=false"], | |
| 186 | + ["exec", "wrangler", "dev", "-e", "dev", "--ip", "127.0.0.1", "--port", String(port), "--inspector-port", String(await freePort()), "--persist-to", persist, "--var", `PUBLIC_URL:${base}`, "--var", "DEV_AUTH:", "--var", `WAIT_TIMER_MINUTE_MS:${TIMER_MINUTE_MS}`, "--var", `MASTER_KEY:${randomBytes(32).toString("base64")}`, "--var", `OIDC_PRIVATE_KEY:${JSON.stringify(generateKeyPairSync("ec", { namedCurve: "P-256" }).privateKey.export({ format: "jwk" }))}`, "--show-interactive-dev-session=false"], | |
| 185 | 187 | { cwd: APP, env: { ...process.env, CI: "1", WRANGLER_SEND_METRICS: "false" }, stdio: ["ignore", "pipe", "pipe"], detached: true }, |
| 186 | 188 | ); |
| 187 | 189 | dev.stdout.on("data", (d) => (devLog += d)); |
| @@ -2266,16 +2268,10 @@ | ||
| 2266 | 2268 | assert.equal(timer.wait_timer, 1); |
| 2267 | 2269 | assert.equal(timer.current_user_can_approve, false); |
| 2268 | 2270 | await status(422, "POST", `${R}/actions/runs/${run3.id}/pending_deployments`, { environment_ids: [timer.environment.id], state: "approved" }, T.rita); |
| 2269 | - const started = Date.now(); | |
| 2270 | - // Real time passes here, so give the run's Workflow room on a busy machine: sit out the timer, | |
| 2271 | - // then check the job every few seconds rather than polling the runner queue throughout. | |
| 2272 | - const due = Date.parse(timer.wait_timer_started_at) + 60_000; | |
| 2273 | - await sleep(Math.max(0, due - Date.now())); | |
| 2274 | - for (let end = Date.now() + 300_000; Date.now() < end; await sleep(3_000)) { | |
| 2275 | - if ((await ok("GET", `${R}/actions/runs/${run3.id}/jobs`)).jobs[0].status !== "waiting") break; | |
| 2276 | - } | |
| 2277 | - const timed = await until("the job after its wait timer", claim, 30_000); | |
| 2278 | - assert.ok(Date.now() - started > 20_000, "the job waited for its timer"); | |
| 2271 | + // The Worker runs with a short wait-timer minute (WAIT_TIMER_MINUTE_MS), so the timer passes in seconds. | |
| 2272 | + if (Date.now() < Date.parse(timer.wait_timer_started_at) + TIMER_MINUTE_MS - 2000) assert.equal(await claim(), null, "nothing starts before the timer passes"); | |
| 2273 | + const timed = await until("the job after its wait timer", claim, 60_000); | |
| 2274 | + assert.ok(Date.now() >= Date.parse(timer.wait_timer_started_at) + TIMER_MINUTE_MS - 1000, "the job waited for its timer"); | |
| 2279 | 2275 | assert.equal(timed.secrets.DEPLOY_KEY, "deploy-value"); |
| 2280 | 2276 | }); |
| 2281 | 2277 |
addedapps/diffinery/test/unit/project-endrun.test.ts+95−0View file
| @@ -1,0 +1,95 @@ | ||
| 1 | +import { describe, expect, it } from "vitest"; | |
| 2 | +import { DatabaseSync } from "node:sqlite"; | |
| 3 | +import { ProjectDO, type RunRow, type TaskView } from "../../src/agentic/ProjectDO.js"; | |
| 4 | +import type { Env } from "../../src/env.js"; | |
| 5 | +import { testD1 } from "./stubs/d1.js"; | |
| 6 | + | |
| 7 | +/** The Durable Object's SqlStorage over node:sqlite. */ | |
| 8 | +function sqlStorage() { | |
| 9 | + const db = new DatabaseSync(":memory:"); | |
| 10 | + return { | |
| 11 | + exec(query: string, ...args: unknown[]) { | |
| 12 | + const params = args.map((a) => (a === undefined ? null : a)) as Array<string | number | null>; | |
| 13 | + let rows: unknown[] = []; | |
| 14 | + if (!params.length && query.trim().replace(/;\s*$/, "").includes(";")) db.exec(query); | |
| 15 | + else rows = db.prepare(query).all(...params); | |
| 16 | + return { | |
| 17 | + toArray: () => rows, | |
| 18 | + one: () => { | |
| 19 | + if (rows.length !== 1) throw new Error(`expected one row, got ${rows.length}`); | |
| 20 | + return rows[0]; | |
| 21 | + }, | |
| 22 | + }; | |
| 23 | + }, | |
| 24 | + }; | |
| 25 | +} | |
| 26 | + | |
| 27 | +/** | |
| 28 | + * A project with one task whose run is in its agent step, in a sandbox. Closing the sandbox calls `during`, which | |
| 29 | + * reads (or changes) the project between the run's end and the sandbox's close. | |
| 30 | + */ | |
| 31 | +function fixture(during: (p: ProjectDO) => Promise<void> | void) { | |
| 32 | + const ctx = { storage: { sql: sqlStorage(), setAlarm: async () => {} }, waitUntil: () => {} } as unknown as DurableObjectState; | |
| 33 | + const stub = { idFromName: (n: string) => n }; | |
| 34 | + let p!: ProjectDO; | |
| 35 | + const env = { | |
| 36 | + DB: testD1(), | |
| 37 | + REPO: { ...stub, get: () => ({ closeTaskSandbox: async () => during(p) }) }, | |
| 38 | + SANDBOX: { ...stub, get: () => ({ call: async () => ({ status: 200, body: "{}" }) }) }, | |
| 39 | + } as unknown as Env; | |
| 40 | + p = new ProjectDO(ctx, env); | |
| 41 | + const sql = (ctx.storage as unknown as { sql: ReturnType<typeof sqlStorage> }).sql; | |
| 42 | + const at = new Date().toISOString(); | |
| 43 | + sql.exec( | |
| 44 | + "INSERT INTO tasks (number, id, repo_id, goal, title, runtime, agent_id, agent_login, person_id, person_login, base, branch, state, created_at, updated_at) VALUES (1, 't1', 'r1', 'Do it', 'Do it', 'custom', 'a1', 'bot', 'u1', 'ada', 'main', 'agent/1-do-it', 'running', ?, ?)", | |
| 45 | + at, | |
| 46 | + at, | |
| 47 | + ); | |
| 48 | + sql.exec("INSERT INTO runs (task, seq, kind, prompt, providers, step, sandbox_id, proc, started_at) VALUES (1, 1, 'start', '', '[]', 'agent', 'sb1', 'agent', ?)", at); | |
| 49 | + const run = () => sql.exec("SELECT * FROM runs WHERE task = 1 AND seq = 1").one() as RunRow; | |
| 50 | + const end = (step: "done" | "failed" | "stopped") => (p as unknown as { endRun(r: RunRow, s: string, e: string | null): Promise<void> }).endRun(run(), step, null); | |
| 51 | + return { p, sql, end }; | |
| 52 | +} | |
| 53 | + | |
| 54 | +describe("a run's end", () => { | |
| 55 | + it("settles its task before the sandbox closes", async () => { | |
| 56 | + let seen: TaskView | null = null; | |
| 57 | + const f = fixture((p) => { | |
| 58 | + seen = p.get(1); | |
| 59 | + }); | |
| 60 | + await f.end("done"); | |
| 61 | + // Read between the run's end and the sandbox's close: the run is done, and so is the task. | |
| 62 | + expect(seen!.runs[0]!.step).toBe("done"); | |
| 63 | + expect(seen!.state).toBe("ready"); | |
| 64 | + expect(f.p.get(1)!.state).toBe("ready"); | |
| 65 | + }); | |
| 66 | + | |
| 67 | + it("keeps a waiting message's task running, and starts its run once the sandbox is closed", async () => { | |
| 68 | + let seen: TaskView | null = null; | |
| 69 | + const f = fixture((p) => { | |
| 70 | + seen = p.get(1); | |
| 71 | + }); | |
| 72 | + f.sql.exec("UPDATE tasks SET queued = 'And add a test' WHERE number = 1"); | |
| 73 | + await f.end("done"); | |
| 74 | + expect(seen!.state).toBe("running"); | |
| 75 | + expect(seen!.runs.length).toBe(1); | |
| 76 | + const after = f.p.get(1)!; | |
| 77 | + expect(after.state).not.toBe("ready"); | |
| 78 | + expect(after.runs.map((r) => [r.seq, r.kind, r.message])).toEqual([ | |
| 79 | + [1, "start", null], | |
| 80 | + [2, "message", "And add a test"], | |
| 81 | + ]); | |
| 82 | + expect(after.queued).toBeNull(); | |
| 83 | + }); | |
| 84 | + | |
| 85 | + it("leaves a stop made while the sandbox closed as it is", async () => { | |
| 86 | + const f = fixture(async (p) => { | |
| 87 | + await p.stop(1, "ada"); | |
| 88 | + }); | |
| 89 | + f.sql.exec("UPDATE tasks SET queued = 'And add a test' WHERE number = 1"); | |
| 90 | + await f.end("done"); | |
| 91 | + const after = f.p.get(1)!; | |
| 92 | + expect(after.state).toBe("stopped"); | |
| 93 | + expect(after.runs.length).toBe(1); | |
| 94 | + }); | |
| 95 | +}); |
addedapps/diffinery/test/unit/stubs/cloudflare-workers.ts+7−0View file
| @@ -1,0 +1,7 @@ | ||
| 1 | +/** Stand-in for the Workers runtime's `cloudflare:workers` module, for unit tests that construct a Durable Object. */ | |
| 2 | +export class DurableObject<Env = unknown> { | |
| 3 | + constructor( | |
| 4 | + protected ctx: DurableObjectState, | |
| 5 | + protected env: Env, | |
| 6 | + ) {} | |
| 7 | +} |
addedapps/diffinery/test/unit/wait-timer.test.ts+23−0View file
| @@ -1,0 +1,23 @@ | ||
| 1 | +import { describe, expect, it } from "vitest"; | |
| 2 | +import { timerMinute } from "../../src/repo/actions.js"; | |
| 3 | + | |
| 4 | +const local = { WAIT_TIMER_MINUTE_MS: "10000", ACCESS_AUD: "", PUBLIC_URL: "http://127.0.0.1:41234" }; | |
| 5 | + | |
| 6 | +describe("an environment's wait-timer minute", () => { | |
| 7 | + it("is a real minute unless the end-to-end setting is there", () => { | |
| 8 | + expect(timerMinute({ ...local, WAIT_TIMER_MINUTE_MS: undefined })).toBe(60_000); | |
| 9 | + expect(timerMinute(local)).toBe(10_000); | |
| 10 | + expect(timerMinute({ ...local, PUBLIC_URL: "http://localhost:8787" })).toBe(10_000); | |
| 11 | + }); | |
| 12 | + | |
| 13 | + it("ignores the setting on a deployed Worker", () => { | |
| 14 | + expect(timerMinute({ ...local, PUBLIC_URL: "https://diffinery.com" })).toBe(60_000); | |
| 15 | + expect(timerMinute({ ...local, PUBLIC_URL: "https://localhost.evil.example" })).toBe(60_000); | |
| 16 | + expect(timerMinute({ ...local, ACCESS_AUD: "aud-tag" })).toBe(60_000); | |
| 17 | + expect(timerMinute({ ...local, PUBLIC_URL: "" })).toBe(60_000); | |
| 18 | + }); | |
| 19 | + | |
| 20 | + it("ignores values that aren't a length from 1 ms to a minute", () => { | |
| 21 | + for (const v of ["0", "-5", "1.5", "60001", "soon", ""]) expect(timerMinute({ ...local, WAIT_TIMER_MINUTE_MS: v })).toBe(60_000); | |
| 22 | + }); | |
| 23 | +}); |
modifiedapps/diffinery/vitest.config.ts+6−1View file
| @@ -5,7 +5,12 @@ | ||
| 5 | 5 | const PREFIX = "virtual:wasm-module/"; |
| 6 | 6 | |
| 7 | 7 | export default defineConfig({ |
| 8 | - resolve: { alias: { "cloudflare:workflows": fileURLToPath(new URL("./test/unit/stubs/cloudflare-workflows.ts", import.meta.url)) } }, | |
| 8 | + resolve: { | |
| 9 | + alias: { | |
| 10 | + "cloudflare:workflows": fileURLToPath(new URL("./test/unit/stubs/cloudflare-workflows.ts", import.meta.url)), | |
| 11 | + "cloudflare:workers": fileURLToPath(new URL("./test/unit/stubs/cloudflare-workers.ts", import.meta.url)), | |
| 12 | + }, | |
| 13 | + }, | |
| 9 | 14 | plugins: [ |
| 10 | 15 | { |
| 11 | 16 | // `import mod from "x.wasm?module"` gives a compiled WebAssembly.Module, like wrangler's CompiledWasm rule in the Worker. |