Diffinery

codician/diffinerypublic

Open-source, agent-first GitHub and GitHub Actions alternative that runs on your own Cloudflare account: git hosting, pull requests, code review and drop-in GitHub Actions YAML on Workers, Durable Objects, R2 and Containers.

Actions keep running when GitHub is down: Diffinery keeps a copy of every action jobs use

Runners still download `uses:` actions from GitHub first. After a download, Diffinery
fetches the same archive from codeload.github.com itself and stores it in R2
(action-mirror/, refreshed daily for tags and branches, weekly for SHAs, 30-day lifecycle
rule). When the GitHub download and the git fallback both fail, the runner unpacks
Diffinery's copy and logs a warning saying when it was fetched. Runners never upload a
copy, so a job can't plant an action for other repositories.

The tarball download now gives up on an attempt that gets no answer in 60 s, and the git
fallback stops after 5 minutes, so a hanging GitHub doesn't hold a job.

Docs: compatibility "When GitHub is down" (what still depends on GitHub), runners.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FL6rJXsUCeA4ZVKUGfZUkj
test-bot authored signed1 parent be0307ecommit bfcc3854f53105ab4627f8903fc9976581d8f057Browse files
12 files changed+440−24
Files (12)
modifiedapps/diffinery/docs/actions/compatibility.md+24−2View file
@@ -18,7 +18,7 @@
1818 | `actions/checkout`, `actions/setup-node`, `actions/setup-python`, `actions/setup-go` | Built into the runner for speed; the real action runs for versions or inputs the built-in doesn't cover | [Built-in actions](#built-in-actions) |
1919 | `actions/cache`, `actions/upload-artifact`, `actions/download-artifact` | Built into the runner, sharing storage and formats with the official actions, which also work | [Caching and artifacts](/docs/actions/caching-and-artifacts) |
2020 | `actions/github-script` | Supported (the real action runs) | [actions/github-script](#actionsgithub-script) |
21-| Other JavaScript, composite and Docker actions | Supported, downloaded from GitHub, with the cache and artifact runtime services | [Other actions](#other-actions) |
21+| Other JavaScript, composite and Docker actions | Supported, downloaded from GitHub, with the cache and artifact runtime services. Diffinery keeps a copy of each one jobs use, for when GitHub is down | [Other actions](#other-actions) |
2222 | Job containers and service containers | Supported | [Docker in workflows](/docs/actions/docker) |
2323 | Reusable workflows | Supported, except a matrix on the calling job | [Reusable workflows](/docs/actions/reusable-workflows) |
2424 | Secrets and variables | Repository, organization and environment secrets and variables | [Secrets and variables](/docs/actions/secrets-and-variables) |
@@ -255,7 +255,7 @@
255255
256256 ## Other actions
257257
258-Any other `uses: owner/repo[/path]@ref` (and a built-in action the built-in doesn't cover) is downloaded from GitHub (`codeload.github.com`, or with `git` from `github.com` when that endpoint can't be reached) when the step runs and executed according to its `action.yml` or `action.yaml`.
258+Any other `uses: owner/repo[/path]@ref` (and a built-in action the built-in doesn't cover) is downloaded from GitHub (`codeload.github.com`, or with `git` from `github.com` when that endpoint can't be reached) when the step runs and executed according to its `action.yml` or `action.yaml`. If GitHub can't be reached at all, the runner uses the copy Diffinery keeps; see [When GitHub is down](#when-github-is-down).
259259
260260 ### JavaScript actions
261261
@@ -291,6 +291,28 @@
291291
292292 `ACTIONS_READ_TOKEN` is a personal or agent token with read access to `acme/my-action`, stored as a secret, because the job token can't read other private repositories. If `acme/my-action` is public, leave `token` out: the job token can read public repositories.
293293
294+### When GitHub is down
295+
296+Diffinery keeps its own copy of every public action that jobs on Diffinery download, so jobs keep running through a GitHub outage:
297+
298+1. The runner always downloads the action from GitHub first, as above, so a job gets the same code it would get on GitHub.
299+2. After the download, Diffinery fetches the same archive from `codeload.github.com` itself and stores it. Copies are refreshed when a job uses them: a tag or branch (`@v4`, `@main`) once it is a day old, a commit SHA once a week. Copies that jobs stop using are deleted after about a month.
300+3. When GitHub fails (the download and the `git` fallback both fail, for example with `HTTP 503`, a refused connection or no answer in 60 seconds), the runner downloads Diffinery's copy instead. The step logs a warning, also shown as an annotation on the run:
301+
302+ ```
303+ GitHub is unreachable (HTTP 503 downloading https://codeload.github.com/<owner>/<repo>/tar.gz/<ref>); using Diffinery's copy of '<owner>/<repo>@<ref>', downloaded from GitHub at <time>
304+ ```
305+
306+Diffinery only stores what it downloaded from GitHub itself, never what a runner sends, so no job can change the action other repositories run. Copies are shared by all repositories, like the public actions they hold. During an outage, a tag or branch runs the version Diffinery last fetched, which can be up to a day older than GitHub's (pin a commit SHA to rule that out). The first job that ever uses an action needs GitHub, and so do actions in private GitHub repositories and archives larger than 256 MB.
307+
308+These still depend on GitHub while it is down:
309+
310+- Anything a step or an action downloads from GitHub at run time, such as release assets from `github.com/<owner>/<repo>/releases/download/...` or `raw.githubusercontent.com` files. Diffinery doesn't proxy or cache those. To keep a job independent of them, cache what you download with `actions/cache` (stored on Diffinery, so a cache hit doesn't touch GitHub).
311+- `actions/setup-python` for a version other than the preinstalled Python 3.12: the builds come from GitHub's `actions/python-versions` releases.
312+- Container images on `ghcr.io`.
313+
314+These don't need GitHub: the code itself (checked out from Diffinery), `actions/cache` and artifacts (stored on Diffinery), the [built-in](#built-in-actions) `actions/setup-node` (the preinstalled Node.js 22, or `nodejs.org`) and `actions/setup-go` (`go.dev`), and the tools preinstalled on hosted runners.
315+
294316 ### Unsupported action runtimes
295317
296318 Any other `runs.using` value fails the step with `Unsupported action runtime 'runs.using: <value>' in <action>`.
modifiedapps/diffinery/docs/actions/runners.md+1−1View file
@@ -376,7 +376,7 @@
376376 - The workspace is `<work dir>/<repository name>` and is **not** wiped between jobs. `actions/checkout` cleans it by default (`clean: true`).
377377 - The tool cache (`RUNNER_TOOL_CACHE`) is `<work dir>/_tool` unless the runner's environment sets `RUNNER_TOOL_CACHE`. Tools that `setup-node`, `setup-python` and `setup-go` download stay there for later jobs.
378378 - `runner.os` reports `Linux` and `RUNNER_ENVIRONMENT` reports `diffinery-hosted` on every machine, including Macs. Use your own labels, or check `uname` in a step, if a workflow needs to know the platform.
379-- Remote actions (`uses: owner/repo@ref`) are downloaded from `codeload.github.com`. Set `DIFFINERY_GITHUB_TOKEN` in the runner's environment to authenticate those downloads (useful against GitHub's rate limits), or `DIFFINERY_ACTIONS_DOWNLOAD_URL` to a URL template containing `{owner}`, `{repo}` and `{ref}` to download from a mirror instead. If the download from `codeload.github.com` fails, the agent tries a shallow `git fetch` of the ref from `https://github.com/<owner>/<repo>` before failing the step with `Failed to download action '<owner>/<repo>@<ref>': ...`. There is no `git` fallback when `DIFFINERY_ACTIONS_DOWNLOAD_URL` is set.
379+- Remote actions (`uses: owner/repo@ref`) are downloaded from `codeload.github.com`. Set `DIFFINERY_GITHUB_TOKEN` in the runner's environment to authenticate those downloads (useful against GitHub's rate limits), or `DIFFINERY_ACTIONS_DOWNLOAD_URL` to a URL template containing `{owner}`, `{repo}` and `{ref}` to download from a mirror instead. If the download from `codeload.github.com` fails, the agent tries a shallow `git fetch` of the ref from `https://github.com/<owner>/<repo>`, then the copy Diffinery keeps of every action jobs have used, as hosted runners do (see [When GitHub is down](/docs/actions/compatibility#when-github-is-down)), before failing the step with `Failed to download action '<owner>/<repo>@<ref>': ...`. There is no `git` fallback, and no copy from Diffinery, when `DIFFINERY_ACTIONS_DOWNLOAD_URL` is set.
380380 - The common official actions (`actions/checkout`, `actions/cache`, `actions/upload-artifact`, `actions/download-artifact`, `actions/setup-node`, `actions/setup-python`, `actions/setup-go`) run as built-in implementations when the version and inputs allow it. Set `DIFFINERY_DISABLE_BUILTINS=1` in the runner's environment to always download and run the real actions on that runner. See [Compatibility with GitHub Actions](/docs/actions/compatibility#built-in-actions).
381381 - `setup-node`, `setup-python` and `setup-go` download their tools from `nodejs.org/dist`, the `actions/python-versions` manifest and `go.dev`. Override the sources with `DIFFINERY_NODE_MIRROR`, `DIFFINERY_PYTHON_MANIFEST` and `DIFFINERY_GO_INDEX` in the runner's environment if the machine can't reach them.
382382
modifiedapps/diffinery/src/http/runner.ts+28−0View file
@@ -7,6 +7,7 @@
77 import { BodyTooLarge, putStream, sized, type BlobBucket } from "../lib/blob-store.js";
88 import { getFirst, headFirst } from "../lib/storage-keys.js";
99 import { HttpError } from "../lib/util.js";
10+import { mirrorKey, parseActionRef, refreshMirror, type MirrorBucket } from "../lib/action-mirror.js";
1011 import { parseRunnerInfo } from "../repo/actions.js";
1112 import { CACHE_LIMIT_BYTES } from "../repo/runtime.js";
1213 import { repoStub } from "./common.js";
@@ -130,6 +131,33 @@
130131 return new Response(obj.body, { headers: { "content-type": "application/gzip" } });
131132 });
132133
134+// ---- Diffinery's copies of GitHub actions, for when GitHub is down (lib/action-mirror.ts)
135+
136+/** The runner downloaded this action from GitHub: keep a recent copy of it. */
137+runner.post("/runner/jobs/:jobId/action-mirror", async (c) => {
138+ const jobId = c.req.param("jobId");
139+ const { token, repoId } = jobAuth(c.req.raw, jobId);
140+ const action = parseActionRef((await c.req.json().catch(() => ({}))) as Record<string, unknown>);
141+ if (!action) throw new HttpError(422, "invalid action");
142+ await repoStub(c.env, repoId).requireRunningJob(jobId, token);
143+ const work = refreshMirror(c.env.BUCKET as unknown as MirrorBucket, action);
144+ // The runner doesn't wait for this: its job can end, and drop the request, before a large action is stored.
145+ c.executionCtx.waitUntil(work);
146+ return c.json({ result: await work });
147+});
148+
149+/** GitHub failed: the stored copy of an action, and when it was fetched from GitHub. */
150+runner.get("/runner/jobs/:jobId/action-mirror", async (c) => {
151+ const jobId = c.req.param("jobId");
152+ const { token, repoId } = jobAuth(c.req.raw, jobId);
153+ const action = parseActionRef({ owner: c.req.query("owner"), repo: c.req.query("repo"), ref: c.req.query("ref") });
154+ if (!action) throw new HttpError(422, "invalid action");
155+ await repoStub(c.env, repoId).requireRunningJob(jobId, token);
156+ const obj = await c.env.BUCKET.get(mirrorKey(action));
157+ if (!obj) throw new HttpError(404, "no copy of this action");
158+ return new Response(obj.body, { headers: { "content-type": "application/gzip", "content-length": String(obj.size), "x-diffinery-fetched-at": obj.uploaded.toISOString() } });
159+});
160+
133161 // ---- OIDC id tokens (actions/core getIDToken compatible: GET ?audience= → { value })
134162
135163 const oidc = async (c: AppContext) => {
addedapps/diffinery/src/lib/action-mirror.ts+96−0View file
@@ -1,0 +1,96 @@
1+import { BodyTooLarge, putStream, type BlobBucket } from "./blob-store.js";
2+import { ACTION_MIRROR_ROOT } from "./storage-keys.js";
3+
4+/**
5+ * Diffinery's own copies of the GitHub actions jobs use (`uses: owner/repo@ref`), so a job keeps
6+ * working while GitHub is down. Runners still download actions from GitHub first, exactly as
7+ * before; after a download they tell Diffinery which action they used, and Diffinery fetches the
8+ * same archive from codeload.github.com itself and stores it in R2. Only when GitHub fails does a
9+ * runner ask for the stored copy.
10+ *
11+ * Runners never upload a copy: what is stored always came straight from GitHub, so a job cannot
12+ * plant an action that other repositories would then run. Copies are shared by every repository,
13+ * like the public actions they hold. Only public actions are stored (the fetch has no GitHub
14+ * credentials).
15+ *
16+ * A tag or branch can move, so its copy is fetched again once it is a day old; a commit SHA can't,
17+ * so its copy only needs a fetch every week, which also keeps it clear of the 30-day lifecycle rule
18+ * on the prefix (scripts/bootstrap.sh) while jobs use it.
19+ */
20+
21+/** Largest archive stored (bigger actions keep depending on GitHub alone). */
22+export const MIRROR_MAX_BYTES = 256 * 1024 * 1024;
23+const DAY_MS = 86_400_000;
24+/** How old a copy may get before the next use fetches it again: a moving ref, and a commit SHA. */
25+export const REFRESH_MS = { ref: DAY_MS, sha: 7 * DAY_MS };
26+
27+const NAME = /^[A-Za-z0-9_.-]{1,100}$/;
28+
29+export interface ActionRef {
30+ owner: string;
31+ repo: string;
32+ ref: string;
33+}
34+
35+/** The action named by untrusted input, or null when it is not a valid GitHub owner, repository and ref. */
36+export function parseActionRef(v: { owner?: unknown; repo?: unknown; ref?: unknown }): ActionRef | null {
37+ const { owner, repo, ref } = v;
38+ if (typeof owner !== "string" || typeof repo !== "string" || typeof ref !== "string") return null;
39+ if (!NAME.test(owner) || !NAME.test(repo) || owner.startsWith(".") || repo === "." || repo === "..") return null;
40+ // git's ref name rules, roughly (git check-ref-format), and no control characters.
41+ if (!ref || ref.length > 255 || ref.startsWith("-") || ref.startsWith("/") || ref.endsWith("/") || ref.endsWith(".lock") || /\.\.|\/\/|@\{|[\s~^:?*[\\\x00-\x1f\x7f]/.test(ref)) return null;
42+ return { owner, repo, ref };
43+}
44+
45+const isSha = (ref: string) => /^[0-9a-f]{40}$/i.test(ref);
46+
47+/** Where the copy of `a` lives. GitHub names are case-insensitive; refs are not. */
48+export function mirrorKey(a: ActionRef): string {
49+ return `${ACTION_MIRROR_ROOT}${a.owner.toLowerCase()}/${a.repo.toLowerCase()}/${encodeURIComponent(isSha(a.ref) ? a.ref.toLowerCase() : a.ref)}.tar.gz`;
50+}
51+
52+/** Whether a copy uploaded at `uploaded` should be fetched again now. */
53+export function isStale(a: ActionRef, uploaded: Date | undefined, now = Date.now()): boolean {
54+ if (!uploaded) return true;
55+ return now - uploaded.getTime() >= (isSha(a.ref) ? REFRESH_MS.sha : REFRESH_MS.ref);
56+}
57+
58+/** The bucket, whose `head` also says when an object was stored (R2 does). */
59+export type MirrorBucket = Omit<BlobBucket, "head"> & { head(key: string): Promise<{ size: number; etag: string; uploaded?: Date } | null> };
60+
61+export type RefreshResult = "fresh" | "stored" | "not-found" | "too-large" | "failed";
62+
63+/**
64+ * Makes sure Diffinery has a recent copy of `a`, fetching it from GitHub when it has none or the
65+ * one it has is stale. Never throws: a failure leaves any older copy in place.
66+ */
67+export async function refreshMirror(bucket: MirrorBucket, a: ActionRef, o: { fetch?: typeof fetch; now?: number; codeload?: string } = {}): Promise<RefreshResult> {
68+ // Jobs that report the same action at once may each fetch it (no promise is shared between
69+ // requests in a Worker); the copies are the same, and the last one stored stays.
70+ const key = mirrorKey(a);
71+ try {
72+ const head = await bucket.head(key);
73+ if (head && !isStale(a, head.uploaded, o.now)) return "fresh";
74+ const base = o.codeload ?? "https://codeload.github.com";
75+ const res = await (o.fetch ?? fetch)(`${base}/${a.owner}/${a.repo}/tar.gz/${encodeURIComponent(a.ref)}`, { headers: { "user-agent": "diffinery-action-mirror" }, redirect: "follow" });
76+ if (res.status === 404) {
77+ await res.body?.cancel();
78+ return "not-found";
79+ }
80+ if (!res.ok || !res.body) {
81+ await res.body?.cancel();
82+ return "failed";
83+ }
84+ const len = Number(res.headers.get("content-length") ?? NaN);
85+ if (len > MIRROR_MAX_BYTES) {
86+ await res.body.cancel();
87+ return "too-large";
88+ }
89+ await putStream(bucket as BlobBucket, key, res.body, MIRROR_MAX_BYTES, { httpMetadata: { contentType: "application/gzip" } });
90+ return "stored";
91+ } catch (e) {
92+ if (e instanceof BodyTooLarge) return "too-large";
93+ console.error("action mirror refresh failed", `${a.owner}/${a.repo}@${a.ref}`, e);
94+ return "failed";
95+ }
96+}
modifiedapps/diffinery/src/lib/storage-keys.ts+4−0View file
@@ -14,6 +14,7 @@
1414 * | `archives/` | zipball/tarball cache (rebuilt on a miss) | deleted after 30 days |
1515 * | `exports/` | account exports (accounts/export.ts) | deleted after 8 days |
1616 * | `events/` | the event archive | deleted after 400 days |
17+ * | `action-mirror/` | copies of the GitHub actions jobs use | deleted after 30 days |
1718 *
1819 * Every prefix also aborts multipart uploads left unfinished for a day.
1920 *
@@ -27,6 +28,9 @@
2728
2829 export const STAGING_ROOT = "actions-staging/";
2930
31+/** Diffinery's copies of the GitHub actions jobs use, shared by every repository (lib/action-mirror.ts). */
32+export const ACTION_MIRROR_ROOT = "action-mirror/";
33+
3034 /** A push body staged in R2 (see {@link pushBodyKey}). */
3135 export type SpilledBody = { spilled: string; size: number };
3236
modifiedapps/diffinery/src/repo/RepoDO.ts+3−0View file
@@ -1558,6 +1558,9 @@
15581558 artifactKeysFor(...a: Parameters<Actions["artifactKeysFor"]>) {
15591559 return this.actions.artifactKeysFor(...a);
15601560 }
1561+ requireRunningJob(...a: Parameters<Actions["requireRunningJob"]>) {
1562+ return this.actions.requireRunningJob(...a);
1563+ }
15611564 artifactQuota() {
15621565 return this.runtime.artifactQuota;
15631566 }
modifiedapps/diffinery/src/repo/actions.ts+6−0View file
@@ -2373,6 +2373,12 @@
23732373 return this.jobRow(jobId);
23742374 }
23752375
2376+ /** Throws unless `token` is the token of job `jobId` and the job is running (the shared action mirror's routes). */
2377+ async requireRunningJob(jobId: string, token: string): Promise<void> {
2378+ const job = await this.authJob(jobId, token);
2379+ if (job.status !== "in_progress") throw new HttpError(409, "job is not running");
2380+ }
2381+
23762382 /** Claims for an OIDC id token; side-effect free (unlike runnerSpec). */
23772383 async oidcClaims(jobId: string, token: string): Promise<{ allowed: boolean; github: Record<string, string> }> {
23782384 const job = await this.authJob(jobId, token);
addedapps/diffinery/test/unit/action-mirror.test.ts+131−0View file
@@ -1,0 +1,131 @@
1+import { describe, expect, it } from "vitest";
2+import { isStale, mirrorKey, parseActionRef, refreshMirror, REFRESH_MS, type MirrorBucket } from "../../src/lib/action-mirror.js";
3+
4+/** An in-memory bucket that remembers when each object was stored. */
5+function memoryBucket(now: () => number) {
6+ const objects = new Map<string, { data: Uint8Array; uploaded: Date }>();
7+ const bytes = async (v: ReadableStream | ArrayBuffer | ArrayBufferView) => new Uint8Array(await new Response(v as BodyInit).arrayBuffer());
8+ const bucket: MirrorBucket = {
9+ async get(key) {
10+ const o = objects.get(key);
11+ return o ? { body: new Response(o.data).body!, size: o.data.length } : null;
12+ },
13+ async head(key) {
14+ const o = objects.get(key);
15+ return o ? { size: o.data.length, etag: "e", uploaded: o.uploaded } : null;
16+ },
17+ async put(key, value) {
18+ const data = await bytes(value);
19+ objects.set(key, { data, uploaded: new Date(now()) });
20+ return { size: data.length, etag: "e" };
21+ },
22+ async list() {
23+ return { objects: [], truncated: false };
24+ },
25+ async delete() {},
26+ async createMultipartUpload(key) {
27+ const parts = new Map<number, Uint8Array>();
28+ return {
29+ uploadId: "u",
30+ async uploadPart(n, v) {
31+ parts.set(n, await bytes(v));
32+ return { partNumber: n, etag: String(n) };
33+ },
34+ async complete(list) {
35+ const data = new Uint8Array(await new Blob(list.map((p) => parts.get(p.partNumber)! as BlobPart)).arrayBuffer());
36+ objects.set(key, { data, uploaded: new Date(now()) });
37+ },
38+ async abort() {},
39+ };
40+ },
41+ resumeMultipartUpload() {
42+ throw new Error("unused");
43+ },
44+ };
45+ return { bucket, objects };
46+}
47+
48+const T0 = Date.parse("2026-10-26T12:00:00Z");
49+
50+describe("action mirror", () => {
51+ it("accepts GitHub action names and refs, and nothing that could escape the key or the URL", () => {
52+ expect(parseActionRef({ owner: "actions", repo: "setup-java", ref: "v4" })).toEqual({ owner: "actions", repo: "setup-java", ref: "v4" });
53+ expect(parseActionRef({ owner: "a", repo: "b", ref: "releases/v1.2" })).not.toBeNull();
54+ for (const bad of [
55+ { owner: "../x", repo: "b", ref: "v1" },
56+ { owner: "a", repo: "..", ref: "v1" },
57+ { owner: "a/b", repo: "c", ref: "v1" },
58+ { owner: "a", repo: "b", ref: "../../etc" },
59+ { owner: "a", repo: "b", ref: "-x" },
60+ { owner: "a", repo: "b", ref: "v1 v2" },
61+ { owner: "a", repo: "b", ref: "" },
62+ { owner: "a", repo: "b" },
63+ { owner: 1, repo: "b", ref: "v1" },
64+ ]) {
65+ expect(parseActionRef(bad), JSON.stringify(bad)).toBeNull();
66+ }
67+ });
68+
69+ it("keys copies by lower-case owner and repository, and keeps refs apart", () => {
70+ expect(mirrorKey({ owner: "Actions", repo: "Checkout", ref: "v4" })).toBe("action-mirror/actions/checkout/v4.tar.gz");
71+ expect(mirrorKey({ owner: "a", repo: "b", ref: "releases/v1" })).toBe("action-mirror/a/b/releases%2Fv1.tar.gz");
72+ const sha = "ABCDEF0123456789ABCDEF0123456789ABCDEF01";
73+ expect(mirrorKey({ owner: "a", repo: "b", ref: sha })).toBe(`action-mirror/a/b/${sha.toLowerCase()}.tar.gz`);
74+ });
75+
76+ it("refreshes a tag after a day and a commit after a week", () => {
77+ const tag = { owner: "a", repo: "b", ref: "v1" };
78+ const sha = { owner: "a", repo: "b", ref: "0123456789abcdef0123456789abcdef01234567" };
79+ expect(isStale(tag, undefined, T0)).toBe(true);
80+ expect(isStale(tag, new Date(T0 - REFRESH_MS.ref + 1000), T0)).toBe(false);
81+ expect(isStale(tag, new Date(T0 - REFRESH_MS.ref), T0)).toBe(true);
82+ expect(isStale(sha, new Date(T0 - REFRESH_MS.ref), T0)).toBe(false);
83+ expect(isStale(sha, new Date(T0 - REFRESH_MS.sha), T0)).toBe(true);
84+ });
85+
86+ it("fetches from GitHub only when its copy is missing or stale, and keeps the old copy when GitHub fails", async () => {
87+ let now = T0;
88+ const { bucket, objects } = memoryBucket(() => now);
89+ const calls: string[] = [];
90+ let answer: () => Response = () => new Response("tarball-1");
91+ const fetchStub = (async (url: string) => {
92+ calls.push(url);
93+ return answer();
94+ }) as unknown as typeof fetch;
95+ const a = { owner: "acme", repo: "tools", ref: "v1" };
96+ const o = { fetch: fetchStub, codeload: "https://codeload.test" };
97+
98+ expect(await refreshMirror(bucket, a, { ...o, now })).toBe("stored");
99+ expect(calls).toEqual(["https://codeload.test/acme/tools/tar.gz/v1"]);
100+ expect(new TextDecoder().decode(objects.get(mirrorKey(a))!.data)).toBe("tarball-1");
101+
102+ now += 3600_000;
103+ expect(await refreshMirror(bucket, a, { ...o, now })).toBe("fresh");
104+ expect(calls).toHaveLength(1);
105+
106+ now += REFRESH_MS.ref;
107+ answer = () => new Response("down", { status: 503 });
108+ expect(await refreshMirror(bucket, a, { ...o, now })).toBe("failed");
109+ expect(new TextDecoder().decode(objects.get(mirrorKey(a))!.data)).toBe("tarball-1");
110+
111+ answer = () => new Response("tarball-2");
112+ expect(await refreshMirror(bucket, a, { ...o, now })).toBe("stored");
113+ expect(new TextDecoder().decode(objects.get(mirrorKey(a))!.data)).toBe("tarball-2");
114+
115+ answer = () => new Response("nope", { status: 404 });
116+ expect(await refreshMirror(bucket, { ...a, ref: "v9" }, { ...o, now })).toBe("not-found");
117+ expect(objects.has(mirrorKey({ ...a, ref: "v9" }))).toBe(false);
118+
119+ answer = () => {
120+ throw new TypeError("network down");
121+ };
122+ expect(await refreshMirror(bucket, { ...a, ref: "v8" }, { ...o, now })).toBe("failed");
123+ });
124+
125+ it("stores nothing for an archive over the size limit", async () => {
126+ const { bucket, objects } = memoryBucket(() => T0);
127+ const big = (() => new Response("x", { headers: { "content-length": String(1024 * 1024 * 1024) } })) as unknown as typeof fetch;
128+ expect(await refreshMirror(bucket, { owner: "a", repo: "huge", ref: "v1" }, { fetch: big })).toBe("too-large");
129+ expect(objects.size).toBe(0);
130+ });
131+});
modifiedcontainers/runner/src/steps/actions.ts+87−20View file
@@ -68,11 +68,25 @@
6868 throw failure(`Can't find 'action.yml' or 'action.yaml' under '${dir}'. Did you forget to run actions/checkout before running your local action?`);
6969 }
7070
71-/** Downloads `owner/repo@ref` into the action cache and returns its root dir. */
71+/** Where actions come from on GitHub, and the first retry's delay (tests point these at local servers and shorten it). */
72+export const actionDownload = { codeload: "https://codeload.github.com", git: "https://github.com", backoffMs: 1000 };
73+
74+/** How long the tarball download waits for GitHub to answer before it counts as a failed attempt. */
75+const ANSWER_TIMEOUT_MS = 60_000;
76+/** How long the `git fetch` fallback may take. */
77+const GIT_TIMEOUT_MS = 300_000;
78+
79+/**
80+ * Downloads `owner/repo@ref` into the action cache and returns its root dir: from GitHub, and when
81+ * GitHub fails, from the copy Diffinery keeps of every action jobs have used (lib/action-mirror.ts
82+ * on the server). After a download from GitHub the runner tells Diffinery, which refreshes its copy
83+ * by fetching the action from GitHub itself; the job doesn't wait for that.
84+ */
7285 export async function downloadAction(job: JobRunner, io: StepIO, owner: string, repo: string, ref: string): Promise<string> {
7386 const dest = path.join(job.actionsDir, owner, repo, ref);
7487 if (existsSync(path.join(dest, ".diffinery-complete"))) return dest;
75- const tpl = process.env.DIFFINERY_ACTIONS_DOWNLOAD_URL ?? "https://codeload.github.com/{owner}/{repo}/tar.gz/{ref}";
88+ const custom = process.env.DIFFINERY_ACTIONS_DOWNLOAD_URL;
89+ const tpl = custom ?? `${actionDownload.codeload}/{owner}/{repo}/tar.gz/{ref}`;
7690 const url = tpl.replace("{owner}", owner).replace("{repo}", repo).replace("{ref}", encodeURIComponent(ref));
7791 job.logger.line(io.logIndex, `Download action repository '${owner}/${repo}@${ref}'`);
7892 const headers: Record<string, string> = { "user-agent": "diffinery-runner" };
@@ -80,38 +94,91 @@
8094 if (ghToken && url.startsWith("https://codeload.github.com/")) headers.authorization = `token ${ghToken}`;
8195 let lastErr: unknown;
8296 for (let attempt = 0; attempt < 4; attempt++) {
83- if (attempt) await new Promise((r) => setTimeout(r, 1000 * 2 ** attempt));
97+ if (attempt) await new Promise((r) => setTimeout(r, actionDownload.backoffMs * 2 ** attempt));
8498 try {
85- const res = await fetch(url, { headers, redirect: "follow" });
99+ // A GitHub outage can leave connections hanging rather than refused.
100+ const answered = new AbortController();
101+ const timer = setTimeout(() => answered.abort(new Error(`no answer from ${new URL(url).host} in ${ANSWER_TIMEOUT_MS / 1000} s`)), ANSWER_TIMEOUT_MS);
102+ const res = await fetch(url, { headers, redirect: "follow", signal: answered.signal }).finally(() => clearTimeout(timer));
86103 if (res.status === 404) throw failure(`Unable to resolve action '${owner}/${repo}@${ref}', repository or version not found`);
87104 if (res.status === 401 || res.status === 403) {
88105 lastErr = new Error(`HTTP ${res.status} downloading ${url}`);
89106 break;
90107 }
91108 if (!res.ok) throw new Error(`HTTP ${res.status} downloading ${url}`);
92- const buf = Buffer.from(await res.arrayBuffer());
93- const tmp = `${dest}.tmp-${process.pid}-${Date.now()}`;
94- await mkdir(tmp, { recursive: true });
95- const tarFile = `${tmp}.tar.gz`;
96- await writeFile(tarFile, buf);
97- const r = await exec("tar", ["-xzf", tarFile, "-C", tmp, "--strip-components=1"], { capture: true });
98- await rm(tarFile, { force: true });
99- if (r.exitCode !== 0) throw new Error(`Failed to extract action archive: ${r.stderr.trim()}`);
100- await rm(dest, { recursive: true, force: true });
101- await mkdir(path.dirname(dest), { recursive: true });
102- await rename(tmp, dest);
103- await writeFile(path.join(dest, ".diffinery-complete"), "");
109+ await extractAction(dest, Buffer.from(await res.arrayBuffer()));
110+ if (!custom) reportAction(job, owner, repo, ref);
104111 return dest;
105112 } catch (e) {
106113 if (e instanceof StepFailure) throw e;
107114 lastErr = e;
108115 }
109116 }
110- // The tarball endpoint can be unreachable (proxies, rate limits) while git still works.
111- if (!process.env.DIFFINERY_ACTIONS_DOWNLOAD_URL && (await cloneAction(dest, owner, repo, ref, ghToken))) return dest;
117+ if (!custom) {
118+ // The tarball endpoint can be unreachable (proxies, rate limits) while git still works.
119+ if (await cloneAction(dest, owner, repo, ref, ghToken)) {
120+ reportAction(job, owner, repo, ref);
121+ return dest;
122+ }
123+ // GitHub is down: Diffinery's copy, if a job has used this action before.
124+ const fetched = await mirroredAction(job, dest, owner, repo, ref);
125+ if (fetched) {
126+ const msg = `GitHub is unreachable (${errMsg(lastErr)}); using Diffinery's copy of '${owner}/${repo}@${ref}', downloaded from GitHub at ${fetched}`;
127+ job.logger.line(io.logIndex, `##[warning]${msg}`);
128+ job.addAnnotation(io.logIndex, { level: "warning", message: msg });
129+ return dest;
130+ }
131+ }
112132 throw failure(`Failed to download action '${owner}/${repo}@${ref}': ${errMsg(lastErr)}`);
113133 }
114134
135+/** Unpacks a GitHub source tarball (one top-level directory) into `dest`. */
136+async function extractAction(dest: string, tarball: Buffer): Promise<void> {
137+ const tmp = `${dest}.tmp-${process.pid}-${Date.now()}`;
138+ await mkdir(tmp, { recursive: true });
139+ try {
140+ const tarFile = `${tmp}.tar.gz`;
141+ await writeFile(tarFile, tarball);
142+ const r = await exec("tar", ["-xzf", tarFile, "-C", tmp, "--strip-components=1"], { capture: true });
143+ await rm(tarFile, { force: true });
144+ if (r.exitCode !== 0) throw new Error(`Failed to extract action archive: ${r.stderr.trim()}`);
145+ await rm(dest, { recursive: true, force: true });
146+ await mkdir(path.dirname(dest), { recursive: true });
147+ await rename(tmp, dest);
148+ await writeFile(path.join(dest, ".diffinery-complete"), "");
149+ } catch (e) {
150+ await rm(tmp, { recursive: true, force: true });
151+ throw e;
152+ }
153+}
154+
155+/** Asks Diffinery to keep a fresh copy of an action this job got from GitHub. In the background: never fails or delays the job. */
156+function reportAction(job: JobRunner, owner: string, repo: string, ref: string): void {
157+ if (!job.api.online) return;
158+ job.api.request("/action-mirror", { method: "POST", json: { owner, repo, ref }, retries: 0 }).then(
159+ (res) => res.body?.cancel(),
160+ () => {},
161+ );
162+}
163+
164+/** Diffinery's copy of the action, unpacked into `dest`: when it was downloaded from GitHub, or null when there is none. */
165+async function mirroredAction(job: JobRunner, dest: string, owner: string, repo: string, ref: string): Promise<string | null> {
166+ if (!job.api.online) return null;
167+ try {
168+ const q = new URLSearchParams({ owner, repo, ref });
169+ const res = await job.api.request(`/action-mirror?${q}`, { retries: 2, allow: [404] });
170+ if (res.status === 404) {
171+ await res.body?.cancel();
172+ return null;
173+ }
174+ const fetched = res.headers.get("x-diffinery-fetched-at") ?? "an earlier time";
175+ await extractAction(dest, Buffer.from(await res.arrayBuffer()));
176+ return fetched;
177+ } catch {
178+ return null;
179+ }
180+}
181+
115182 /** Fetches `owner/repo@ref` (branch, tag or commit SHA) from github.com with git; false when that fails. */
116183 async function cloneAction(dest: string, owner: string, repo: string, ref: string, token: string | undefined): Promise<boolean> {
117184 if (!/^[\w.-]+$/.test(owner) || !/^[\w.-]+$/.test(repo) || ref.startsWith("-") || /[\s\\]|\.\.|[~^:?*[]/.test(ref)) return false;
@@ -121,10 +188,10 @@
121188 const auth: Record<string, string> = token
122189 ? { GIT_CONFIG_COUNT: String(n + 1), [`GIT_CONFIG_KEY_${n}`]: "http.https://github.com/.extraheader", [`GIT_CONFIG_VALUE_${n}`]: `AUTHORIZATION: basic ${Buffer.from(`x-access-token:${token}`).toString("base64")}` }
123190 : {};
124- const git = (args: string[]) => exec("git", args, { cwd: tmp, capture: true, env: { ...process.env, GIT_TERMINAL_PROMPT: "0", ...auth } });
191+ const git = (args: string[]) => exec("git", args, { cwd: tmp, capture: true, timeoutMs: GIT_TIMEOUT_MS, env: { ...process.env, GIT_TERMINAL_PROMPT: "0", ...auth } });
125192 try {
126193 await mkdir(tmp, { recursive: true });
127- for (const args of [["init", "-q"], ["fetch", "-q", "--depth", "1", `https://github.com/${owner}/${repo}`, ref], ["checkout", "-q", "FETCH_HEAD"]]) {
194+ for (const args of [["init", "-q"], ["fetch", "-q", "--depth", "1", `${actionDownload.git}/${owner}/${repo}`, ref], ["checkout", "-q", "FETCH_HEAD"]]) {
128195 if ((await git(args)).exitCode !== 0) throw new Error(args[0]);
129196 }
130197 await rm(path.join(tmp, ".git"), { recursive: true, force: true });
modifiedcontainers/runner/test/helpers.ts+16−0View file
@@ -26,6 +26,9 @@
2626 /** Fail the next N log POSTs with 503 (to test retries). */
2727 failLogs: number;
2828 specs: Map<string, JobSpec>;
29+ /** Diffinery's copies of actions, by `owner/repo@ref`, and the actions the runner reported downloading from GitHub. */
30+ mirror: Map<string, Buffer>;
31+ mirrorReports: string[];
2932 close(): Promise<void>;
3033 logText(step?: number): string;
3134 lastReport(index: number): StepReport | undefined;
@@ -47,6 +50,8 @@
4750 requests: [],
4851 failLogs: 0,
4952 specs: new Map(),
53+ mirror: new Map(),
54+ mirrorReports: [],
5055 close: async () => {},
5156 logText(step?: number) {
5257 return this.logs
@@ -196,6 +201,17 @@
196201 api.completion = JSON.parse(body.toString()) as JobCompletion;
197202 return json(200, {});
198203 }
204+ if (sub === "/action-mirror" && req.method === "POST") {
205+ const a = JSON.parse(body.toString()) as { owner: string; repo: string; ref: string };
206+ api.mirrorReports.push(`${a.owner}/${a.repo}@${a.ref}`);
207+ return json(200, { result: "stored" });
208+ }
209+ if (sub === "/action-mirror" && req.method === "GET") {
210+ const copy = api.mirror.get(`${u.searchParams.get("owner")}/${u.searchParams.get("repo")}@${u.searchParams.get("ref")}`);
211+ if (!copy) return json(404, { message: "no copy of this action" });
212+ res.writeHead(200, { "content-type": "application/gzip", "x-diffinery-fetched-at": "2026-10-25T12:00:00.000Z" });
213+ return res.end(copy);
214+ }
199215 if (sub === "/oidc") return json(200, { value: `oidc-for-${u.searchParams.get("audience")}` });
200216 return json(404, {});
201217 });
modifiedcontainers/runner/test/job.test.ts+42−0View file
@@ -8,6 +8,7 @@
88 import type { JobSpec, StepSpec } from "@diffinery/shared";
99 import { apiBaseFor, runAgentJob } from "../src/agent.js";
1010 import type { JobRunner } from "../src/job.js";
11+import { actionDownload } from "../src/steps/actions.js";
1112 import { makeSpec, startFakeApi, tmp, type FakeApi } from "./helpers.js";
1213
1314 let api: FakeApi;
@@ -420,6 +421,47 @@
420421 }
421422 });
422423
424+ it("reports actions it got from GitHub, and uses Diffinery's copy when GitHub is down", async () => {
425+ const src = tmp("mirror-action-");
426+ mkdirSync(path.join(src, "tools-abc123"), { recursive: true });
427+ writeFileSync(path.join(src, "tools-abc123/action.yml"), "name: hello\nruns:\n using: node20\n main: main.js\n");
428+ writeFileSync(path.join(src, "tools-abc123/main.js"), 'console.log("action ran:", process.env.GITHUB_ACTION_REPOSITORY, process.env.GITHUB_ACTION_REF);');
429+ const tarball = path.join(src, "a.tgz");
430+ execFileSync("tar", ["-czf", tarball, "-C", src, "tools-abc123"]);
431+ // A fake codeload that serves acme/up and is down for everything else.
432+ const hits: string[] = [];
433+ const server = createServer((req, res) => {
434+ hits.push(req.url ?? "");
435+ if (req.url === "/acme/up/tar.gz/v1") {
436+ res.writeHead(200, { "content-type": "application/gzip" });
437+ res.end(readFileSync(tarball));
438+ } else {
439+ res.writeHead(503);
440+ res.end();
441+ }
442+ });
443+ await new Promise<void>((r) => server.listen(0, "127.0.0.1", () => r()));
444+ const prev = { ...actionDownload };
445+ Object.assign(actionDownload, { codeload: `http://127.0.0.1:${(server.address() as AddressInfo).port}`, git: "http://127.0.0.1:9", backoffMs: 1 });
446+ api.mirror.set("acme/down@v2", readFileSync(tarball));
447+ try {
448+ const c = await run(makeSpec(api, [{ uses: "acme/up@v1" }, { uses: "acme/down@v2" }, { uses: "acme/gone@v3" }]));
449+ expect(api.logText(0)).toContain("action ran: acme/up v1");
450+ expect(api.logText(1)).toContain("using Diffinery's copy of 'acme/down@v2', downloaded from GitHub at 2026-10-25T12:00:00.000Z");
451+ expect(api.logText(1)).toContain("action ran: acme/down v2");
452+ expect(hits.filter((h) => h === "/acme/down/tar.gz/v2")).toHaveLength(4); // GitHub first, with its retries
453+ expect(c.conclusion).toBe("failure");
454+ expect(c.failure?.step).toBe(2);
455+ expect(c.failure?.message).toContain("Failed to download action 'acme/gone@v3': HTTP 503");
456+ // Only what came from GitHub is reported (Diffinery then fetches it from GitHub itself).
457+ expect(api.mirrorReports).toEqual(["acme/up@v1"]);
458+ } finally {
459+ Object.assign(actionDownload, prev);
460+ await new Promise<void>((r) => server.close(() => r()));
461+ rmSync(src, { recursive: true, force: true });
462+ }
463+ });
464+
423465 it("runs github-script, and built-ins it cannot mirror, as the real actions", async () => {
424466 // fake codeload serving one node action for every repository
425467 const src = tmp("real-action-");
modifiedscripts/bootstrap.sh+2−1View file
@@ -70,7 +70,8 @@
7070 {"id":"actions-staging-1d","enabled":true,"conditions":{"prefix":"actions-staging/"},"deleteObjectsTransition":{"condition":{"type":"Age","maxAge":86400}}},
7171 {"id":"archives-30d","enabled":true,"conditions":{"prefix":"archives/"},"deleteObjectsTransition":{"condition":{"type":"Age","maxAge":2592000}}},
7272 {"id":"exports-8d","enabled":true,"conditions":{"prefix":"exports/"},"deleteObjectsTransition":{"condition":{"type":"Age","maxAge":691200}}},
73- {"id":"events-400d","enabled":true,"conditions":{"prefix":"events/"},"deleteObjectsTransition":{"condition":{"type":"Age","maxAge":34560000}}}
73+ {"id":"events-400d","enabled":true,"conditions":{"prefix":"events/"},"deleteObjectsTransition":{"condition":{"type":"Age","maxAge":34560000}}},
74+ {"id":"action-mirror-30d","enabled":true,"conditions":{"prefix":"action-mirror/"},"deleteObjectsTransition":{"condition":{"type":"Age","maxAge":2592000}}}
7475 ]}'
7576 # Applied on every deploy; a failure stops it, since a wrong rule can delete data.
7677 RESULT="$(cf PUT "/r2/buckets/diffinery-data/lifecycle" "$LIFECYCLE")"