Diffinery

codician/diffinerypublic

Open-source, agent-first GitHub and GitHub Actions alternative that runs on your own Cloudflare account: git hosting, pull requests, code review and drop-in GitHub Actions YAML on Workers, Durable Objects, R2 and Containers.

A newer run replaces older runs waiting in its concurrency group, like GitHub (#251)

* A newer run replaces older runs waiting in its concurrency group, like GitHub
test-bot authored 1 parent 961ad5ecommit 661e309ce989968e29581fb653414e53712919faBrowse files
5 files changed+70−5
Files (5)
modifiedapps/diffinery/docs/actions/compatibility.md+1−1View file
@@ -70,7 +70,7 @@
7070 | `jobs.<id>.timeout-minutes` | Default and maximum 360. Larger values are capped at 360. |
7171 | `jobs.<id>.strategy.matrix` | At most 256 jobs per matrix, and at most 1,000 jobs in a run, matrix legs and the jobs of called workflows included. |
7272 | `jobs.<id>.uses` with `strategy.matrix` | Not supported yet. The run fails with `jobs.<id>: a matrix on a job that calls a reusable workflow is not supported yet`. |
73-| `concurrency` (workflow level) | Several runs can wait in a group; GitHub keeps only the newest pending one. Job-level `concurrency` keeps only the newest pending job, as on GitHub. See [Concurrency](/docs/actions/overview#concurrency). |
73+| `concurrency` (workflow level) | Same as GitHub: one run in progress and at most one waiting per group; a newer run cancels older waiting ones. After 6 hours of waiting a run starts anyway. See [Concurrency](/docs/actions/overview#concurrency). |
7474 | `defaults.run.shell`, `jobs.<id>.steps[*].shell` | `cmd` isn't supported. `pwsh` and `powershell` need PowerShell, which hosted runners don't have. |
7575
7676 An unknown key gives a warning such as `unknown job key '<key>'` or `unknown step key '<key>'`, and the key is ignored. The full reference is [Workflow syntax](/docs/actions/workflow-syntax).
modifiedapps/diffinery/docs/actions/overview.md+2−2View file
@@ -405,10 +405,10 @@
405405
406406 - The group is evaluated when the run is created, with the `github`, `inputs` and `vars` contexts. Groups are scoped to the repository, so two workflows can share a group.
407407 - With `cancel-in-progress: true`, creating a run cancels every unfinished run (queued or in progress) in the same group, with the reason `cancelled by a newer run in the same concurrency group`.
408-- Without it, a new run waits while another run in the group is in progress, checking once a minute. After 6 hours of waiting it starts anyway.
408+- Without it, a new run waits while another run in the group is in progress, checking once a minute. After 6 hours of waiting it starts anyway. As on GitHub, a group has at most one waiting run: a new run cancels older runs still waiting in the group (not the one in progress), with the reason `Canceling since a higher priority waiting request for '<group>' exists`, so a deploy group runs the newest push next, never older ones one after another. A run cancelled while it waits finishes within a minute, with none of its jobs run.
409409 - A run or job on a protected branch is cancelled or replaced through its group only by runs on that same branch. A workflow on another branch that names the same group, `cancel-in-progress` or not, waits for it instead, so a branch can't cancel a protected branch's deploy.
410410
411-Differences from GitHub: GitHub keeps at most one pending run per group and cancels older pending ones; Diffinery lets several runs wait and starts them as the group frees up. If the group expression fails to evaluate, the run has no concurrency group.
411+If the group expression fails to evaluate, the run has no concurrency group.
412412
413413 ### Job concurrency
414414
modifiedapps/diffinery/docs/actions/workflow-syntax.md+1−1View file
@@ -220,7 +220,7 @@
220220 cancel-in-progress: true
221221 ```
222222
223-`group` is evaluated when the run is created, with the `github`, `inputs` and `vars` contexts. `cancel-in-progress` is `true`, `false` (the default) or an expression evaluated with the same contexts. With `true`, a new run cancels every unfinished run in its group. With `false`, the new run waits while another run in the group is in progress. Groups are shared by all workflows of the repository, but a run on a protected branch is only cancelled by runs on that same branch; runs from elsewhere wait for it. See [Concurrency](/docs/actions/overview#concurrency) for the differences from GitHub.
223+`group` is evaluated when the run is created, with the `github`, `inputs` and `vars` contexts. `cancel-in-progress` is `true`, `false` (the default) or an expression evaluated with the same contexts. With `true`, a new run cancels every unfinished run in its group. With `false`, the new run waits while another run in the group is in progress, and cancels older runs still waiting there (as on GitHub, a group has one waiting run at most). Groups are shared by all workflows of the repository, but a run on a protected branch is only cancelled by runs on that same branch; runs from elsewhere wait for it. See [Concurrency](/docs/actions/overview#concurrency) for the differences from GitHub.
224224
225225 A value that is neither a string nor a mapping with `group` fails with `concurrency must be a group string or a mapping with 'group'`.
226226
modifiedapps/diffinery/src/repo/actions.ts+24−1View file
@@ -804,7 +804,9 @@
804804 const cancelInProgress = evaluateValue(wf.concurrency.cancelInProgress, ctx);
805805 const others = this.sql.exec<{ id: string; ref: string }>("SELECT id, ref FROM runs WHERE concurrency_group = ? AND status != 'completed' AND id != ?", run.concurrency_group, id).toArray();
806806 // A run the newcomer may not cancel stays, and the newcomer waits for it like any run in the group.
807- for (const other of others) if ((cancelInProgress === true || cancelInProgress === "true") && this.mayCancel(run, other)) await this.cancelInternal(other.id, "cancelled by a newer run in the same concurrency group");
807+ if (cancelInProgress === true || cancelInProgress === "true") {
808+ for (const other of others) if (this.mayCancel(run, other)) await this.cancelInternal(other.id, "cancelled by a newer run in the same concurrency group");
809+ } else await this.replaceWaiting(run);
808810 }
809811 await this.env.RUN_WORKFLOW.create({ id, params: { repoId: this.repo.meta.repoId, runId: id } });
810812 const m = this.repo.meta;
@@ -910,6 +912,13 @@
910912 const { order, errors } = jobOrder(exp.workflow);
911913 if (errors.length) return { jobs: [], error: errors.map((e) => e.message).join("; "), concurrencyBlocked: false };
912914 if (order.flat().length > MAX_RUN_JOBS) return { jobs: [], error: `A run may have at most ${MAX_RUN_JOBS} jobs, those of called workflows included; this one has ${order.flat().length}.`, concurrencyBlocked: false };
915+ if (run.concurrency_group) {
916+ // Runs that waited in the group before newer runs replaced older ones: the newest replaces them now.
917+ const newest = this.sql.exec<RunRow>("SELECT * FROM runs WHERE concurrency_group = ? AND status = 'queued' AND conclusion IS NULL ORDER BY rowid DESC LIMIT 1", run.concurrency_group).toArray()[0];
918+ if (newest && newest.id !== run.id) await this.replaceWaiting(newest);
919+ }
920+ // Cancelled while it waited for its group: none of its jobs will run, so it finishes now instead of waiting its turn.
921+ if (this.runRow(runId).conclusion === "cancelled") return { jobs: [], concurrencyBlocked: false };
913922 const holders = () => (run.concurrency_group ? this.sql.exec<{ n: number }>("SELECT count(*) AS n FROM runs WHERE concurrency_group = ? AND status = 'in_progress' AND id != ?", run.concurrency_group, runId).one().n : 0);
914923 // A run holding the group may be stuck though its jobs all finished (STUCK_RUN_MS): repair it rather than wait behind it.
915924 if (holders()) await this.repairStuckRuns();
@@ -1694,6 +1703,20 @@
16941703 }
16951704
16961705 /**
1706+ * Like GitHub, a concurrency group without cancel-in-progress has at most one run waiting: a newer
1707+ * run takes the place of older ones still waiting, which are cancelled. Otherwise every push to a
1708+ * deploy's branch waits, and once the running deploy ends they start in no set order, each
1709+ * deploying an older commit than the last push's (tactfolio, 2026-10-16: 15 deploys queued
1710+ * behind one that hung, production went back to older commits).
1711+ */
1712+ private async replaceWaiting(run: RunRow): Promise<void> {
1713+ const older = this.sql
1714+ .exec<{ id: string; ref: string }>("SELECT id, ref FROM runs WHERE concurrency_group = ? AND status = 'queued' AND conclusion IS NULL AND id != ? AND rowid < (SELECT rowid FROM runs WHERE id = ?)", run.concurrency_group, run.id, run.id)
1715+ .toArray();
1716+ for (const o of older) if (this.mayCancel(run, o)) await this.cancelInternal(o.id, `Canceling since a higher priority waiting request for '${run.concurrency_group}' exists`);
1717+ }
1718+
1719+ /**
16971720 * Whether a run of `by` may cancel or replace `other` in a shared concurrency group. Groups are
16981721 * repository-wide, as on GitHub, but a workflow pushed to some branch could otherwise name a
16991722 * deploy's group and cancel it halfway: a run on a protected branch is cancelled only by runs on
modifiedapps/diffinery/test/unit/stuck-runs.test.ts+42−0View file
@@ -206,3 +206,45 @@
206206 expect(f.actions.getRun(b).status).toBe("in_progress");
207207 });
208208 });
209+
210+describe("a concurrency group without cancel-in-progress", () => {
211+ it("keeps one run waiting, like GitHub: a newer run replaces older waiting ones, which finish cancelled at once", async () => {
212+ const f = fixture();
213+ const a = await f.push();
214+ expect((await f.actions.plan(a)).concurrencyBlocked).toBe(false);
215+ await f.actions.startJobKey(a, "test");
216+ const b = await f.push();
217+ expect((await f.actions.plan(b)).concurrencyBlocked).toBe(true);
218+ const c = await f.push();
219+ const d = await f.push();
220+ // b and c are replaced as d comes in; the running a is left alone.
221+ expect(f.actions.getRun(b).conclusion).toBe("cancelled");
222+ expect(f.actions.getRun(c).conclusion).toBe("cancelled");
223+ expect(f.actions.getRun(a)).toMatchObject({ status: "in_progress", conclusion: null });
224+ // A replaced run doesn't wait its turn: its next plan has nothing to run, and its workflow finishes it.
225+ expect(await f.actions.plan(c)).toEqual({ jobs: [], concurrencyBlocked: false });
226+ expect((await f.actions.finishRun(c)).conclusion).toBe("cancelled");
227+ expect(f.jobs(c)).toEqual([]);
228+ expect((await f.actions.plan(d)).concurrencyBlocked).toBe(true);
229+ // When a ends, d (the last push) runs next.
230+ await (f.actions as unknown as { completeJob(id: string, c: unknown): Promise<void> }).completeJob(f.jobs(a)[0]!.id, { conclusion: "success", outputs: {} });
231+ await f.actions.startJobKey(a, "deploy");
232+ await (f.actions as unknown as { completeJob(id: string, c: unknown): Promise<void> }).completeJob(f.jobs(a)[1]!.id, { conclusion: "success", outputs: {} });
233+ await f.actions.finishRun(a);
234+ expect((await f.actions.plan(d)).concurrencyBlocked).toBe(false);
235+ expect(f.actions.getRun(d).status).toBe("in_progress");
236+ });
237+
238+ it("clears runs that waited before newer runs replaced older ones: the oldest doesn't take the group ahead of the newest", async () => {
239+ const f = fixture();
240+ const runs = [await f.push(), await f.push(), await f.push()];
241+ // As in tactfolio's deploy group before this: every push waiting, none cancelled.
242+ f.sql.exec("UPDATE runs SET status = 'queued', conclusion = NULL");
243+ // The group is free, and the oldest plans first: it is replaced by the newest, and runs nothing.
244+ expect(await f.actions.plan(runs[0]!)).toEqual({ jobs: [], concurrencyBlocked: false });
245+ expect(f.actions.getRun(runs[1]!).conclusion).toBe("cancelled");
246+ expect(f.actions.getRun(runs[2]!)).toMatchObject({ status: "queued", conclusion: null });
247+ expect((await f.actions.plan(runs[2]!)).concurrencyBlocked).toBe(false);
248+ expect(f.actions.getRun(runs[2]!).status).toBe("in_progress");
249+ });
250+});