Deploy: reuse the runner image unless the runner changed (#54)
scripts/runner-image.sh tags the runner image with a hash of its build inputs and the ISO week, builds and pushes it only when Cloudflare's registry lacks that tag, and render-config points every container at it. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ruyx3Qr4jYrZEpPi466deb
Canberk Demirci authored · GitHub committed signed1 parent c6be185commit 565cd54d2d07bd1b435420bb594f9b9b5d6db9b5Browse files
4 files changed+50−1
Files (4)
modified.github/workflows/deploy.yml+2−0View file
| @@ -56,6 +56,8 @@ | ||
| 56 | 56 | - run: pnpm install --frozen-lockfile |
| 57 | 57 | - name: Build runner agent and web islands |
| 58 | 58 | run: pnpm -r --if-present build |
| 59 | + - name: Runner image (built only when the runner changed) | |
| 60 | + run: scripts/runner-image.sh | |
| 59 | 61 | |
| 60 | 62 | - name: Create or update Cloudflare resources |
| 61 | 63 | id: bootstrap |
modifiedREADME.md+3−1View file
| @@ -63,7 +63,9 @@ | ||
| 63 | 63 | CI (`.github/workflows/ci.yml`: typecheck, unit tests and the Worker bundle) on |
| 64 | 64 | Diffinery's hosted runners. When it passes, `.github/workflows/deploy.yml` |
| 65 | 65 | deploys that commit, followed by the live test (`.github/workflows/live-test.yml`); |
| 66 | -the end-to-end suite (`.github/workflows/e2e.yml`) runs alongside. GitHub stays the upstream for | |
| 66 | +the end-to-end suite (`.github/workflows/e2e.yml`) runs alongside. Deploys | |
| 67 | +reuse the runner image from Cloudflare's registry and rebuild it only when | |
| 68 | +`containers/runner` changed, or weekly for base image updates (`scripts/runner-image.sh`). GitHub stays the upstream for | |
| 67 | 69 | now: `.github/workflows/sync-to-diffinery.yml` copies every branch and tag, and |
| 68 | 70 | the secrets and variables below, to Diffinery on each push, then waits for the |
| 69 | 71 | Actions runs there and fails if any of them fail. Running Deploy by hand on |
modifiedscripts/render-config.mjs+8−0View file
| @@ -51,5 +51,13 @@ | ||
| 51 | 51 | cfg.vars.MAIL_FROM = `notifications@${url.hostname}`; |
| 52 | 52 | } |
| 53 | 53 | if (values["mail-from"]) cfg.vars.MAIL_FROM = values["mail-from"]; |
| 54 | +// A runner image already in the registry (scripts/runner-image.sh): containers use it | |
| 55 | +// instead of building the Dockerfile on every deploy. | |
| 56 | +if (process.env.RUNNER_IMAGE) { | |
| 57 | + for (const c of cfg.containers) { | |
| 58 | + c.image = process.env.RUNNER_IMAGE; | |
| 59 | + delete c.image_build_context; | |
| 60 | + } | |
| 61 | +} | |
| 54 | 62 | writeFileSync(join(appDir, "wrangler.deploy.jsonc"), `// Generated by scripts/render-config.mjs; do not edit.\n${JSON.stringify(cfg, null, 2)}\n`); |
| 55 | 63 | console.log(`wrote ${join(appDir, "wrangler.deploy.jsonc")}`); |
addedscripts/runner-image.sh+37−0View file
| @@ -1,0 +1,37 @@ | ||
| 1 | +#!/usr/bin/env bash | |
| 2 | +# Picks the runner image for a deploy. The tag is a hash of everything the image is built | |
| 3 | +# from (the Dockerfile and its build context: the built agent and image/), plus the ISO | |
| 4 | +# week so base packages are refreshed weekly. If the Cloudflare registry already has that | |
| 5 | +# tag, the deploy reuses it; otherwise it is built and pushed now. Any change to the runner | |
| 6 | +# changes the tag, so a changed runner is always rebuilt. | |
| 7 | +# | |
| 8 | +# Run from the repository root after `pnpm -r build`, with CLOUDFLARE_API_TOKEN and | |
| 9 | +# CLOUDFLARE_ACCOUNT_ID set. Prints the image and, in Actions, sets RUNNER_IMAGE for later | |
| 10 | +# steps (scripts/render-config.mjs points every container at it). | |
| 11 | +set -euo pipefail | |
| 12 | +ctx=containers/runner | |
| 13 | +name=diffinery-runner | |
| 14 | +[ -f "$ctx/dist/agent.mjs" ] || { echo "::error::$ctx/dist/agent.mjs is missing; run pnpm -r build first"; exit 1; } | |
| 15 | + | |
| 16 | +tag=$( | |
| 17 | + { | |
| 18 | + date -u +%G-W%V | |
| 19 | + cd "$ctx" && find Dockerfile .dockerignore dist/agent.mjs image -type f | LC_ALL=C sort | xargs sha256sum | |
| 20 | + } | sha256sum | cut -c1-24 | |
| 21 | +) | |
| 22 | +: "${CLOUDFLARE_ACCOUNT_ID:?CLOUDFLARE_ACCOUNT_ID is required}" | |
| 23 | +image="registry.cloudflare.com/$CLOUDFLARE_ACCOUNT_ID/$name:$tag" | |
| 24 | +wrangler() { (cd apps/diffinery && pnpm exec wrangler "$@"); } | |
| 25 | + | |
| 26 | +if ! listed=$(wrangler containers images list --json --filter "(^|/)$name\$" 2>&1); then | |
| 27 | + echo "::warning::could not list the registry's images, so the runner image is built again: $(tail -n 3 <<<"$listed")" | |
| 28 | + listed="" | |
| 29 | +fi | |
| 30 | +if grep -q "\"$tag\"" <<<"$listed"; then | |
| 31 | + echo "reusing $name:$tag: the runner hasn't changed since it was built" | |
| 32 | +else | |
| 33 | + echo "building $name:$tag" | |
| 34 | + wrangler containers build "$PWD/$ctx" -t "$name:$tag" -p | |
| 35 | +fi | |
| 36 | +[ -z "${GITHUB_ENV:-}" ] || echo "RUNNER_IMAGE=$image" >> "$GITHUB_ENV" | |
| 37 | +echo "$image" |