Diffinery

codician/diffinerypublic

Open-source, agent-first GitHub and GitHub Actions alternative that runs on your own Cloudflare account: git hosting, pull requests, code review and drop-in GitHub Actions YAML on Workers, Durable Objects, R2 and Containers.

Live test imports from GitHub; an import campaign workflow (#38)

The live test imports a small public repository token-free and checks the
default branch, branches, and issue and pull request numbers and titles
against GitHub, and that importing ran no workflows. A GitHub rate limit
skips the check with a warning.

The import campaign (started by hand) imports a varied set of public
repositories as private test repositories, runs their push workflows on
Diffinery and writes a report of what came across and what ran.


Claude-Session: https://claude.ai/code/session_01AizKXSsengXZRegJxRATkL

Co-authored-by: Claude <noreply@anthropic.com>
Canberk Demirci authored · GitHub committed signed1 parent 95a6a14commit be5ab61627c2aa8efdec81e4e563a052628221d7Browse files
3 files changed+294−1
added.github/workflows/import-campaign.yml+92−0View file
@@ -1,0 +1,92 @@
1+name: Import campaign
2+
3+# Imports a varied set of public GitHub repositories into Diffinery as private test
4+# repositories (<owner>/campaign-*), runs their push workflows there, and reports what
5+# came across and what ran (scripts/import-campaign.sh). Started by hand.
6+#
7+# Dispatched on GitHub, it starts the same workflow on Diffinery and prints its report;
8+# on Diffinery it does the work. Imports use the IMPORT_GITHUB_TOKEN secret when it is
9+# set (a read-only token for public repositories), otherwise GitHub's anonymous rate
10+# limit applies and large repositories take hours.
11+on:
12+ workflow_dispatch:
13+ inputs:
14+ repos:
15+ description: GitHub repositories to import, separated by spaces
16+ type: string
17+ default: >-
18+ octocat/git-consortium sindresorhus/slugify expressjs/cors spf13/pflag
19+ pallets/markupsafe BurntSushi/walkdir actions/hello-world-javascript-action
20+ actions/hello-world-docker-action jekyll/minima
21+ minutes:
22+ description: How long to wait for imports and workflow runs, in minutes
23+ type: string
24+ default: "300"
25+
26+permissions:
27+ contents: read
28+
29+concurrency:
30+ group: import-campaign
31+ cancel-in-progress: false
32+
33+jobs:
34+ relay:
35+ name: Start the campaign on Diffinery and print its report
36+ if: ${{ github.server_url == 'https://github.com' }}
37+ runs-on: ubuntu-latest
38+ timeout-minutes: 360
39+ environment: production
40+ env:
41+ DIFFINERY: https://diffinery.com
42+ TOKEN: ${{ secrets.DIFFINERY_LIVE_TOKEN }}
43+ R: codician-team/diffinery
44+ steps:
45+ - name: Dispatch and follow
46+ env:
47+ REPOS: ${{ inputs.repos }}
48+ MINUTES: ${{ inputs.minutes }}
49+ run: |
50+ set -euo pipefail
51+ api() { curl -fsS --retry 3 -X "$1" "$DIFFINERY/api/v3$2" -H "Authorization: Bearer $TOKEN" -H "content-type: application/json" ${3:+--data "$3"}; }
52+ before=$(api GET "/repos/$R/actions/workflows/import-campaign.yml/runs?per_page=1" | jq -r '.workflow_runs[0].id // 0')
53+ api POST "/repos/$R/actions/workflows/import-campaign.yml/dispatches" "$(jq -nc --arg r "$REPOS" --arg m "$MINUTES" '{ref: "main", inputs: {repos: $r, minutes: $m}}')" >/dev/null
54+ for i in $(seq 1 60); do
55+ run=$(api GET "/repos/$R/actions/workflows/import-campaign.yml/runs?per_page=1" | jq -c '.workflow_runs[0] // empty')
56+ [ -n "$run" ] && [ "$(jq -r .id <<<"$run")" != "$before" ] && break
57+ sleep 5
58+ done
59+ id=$(jq -r .id <<<"$run")
60+ echo "Diffinery run: $(jq -r .html_url <<<"$run")"
61+ while [ "$(api GET "/repos/$R/actions/runs/$id" | jq -r .status)" != completed ]; do sleep 30; done
62+ run=$(api GET "/repos/$R/actions/runs/$id")
63+ for job in $(api GET "/repos/$R/actions/runs/$id/jobs" | jq -r '.jobs[].id'); do
64+ curl -fsS "$DIFFINERY/api/v3/repos/$R/actions/jobs/$job/logs" -H "Authorization: Bearer $TOKEN" | sed -n '/^# Import campaign/,$p' | tee -a "$GITHUB_STEP_SUMMARY"
65+ done
66+ [ "$(jq -r .conclusion <<<"$run")" = success ] || { echo "::error::the campaign run on Diffinery ended $(jq -r .conclusion <<<"$run")"; exit 1; }
67+
68+ campaign:
69+ name: Import, run and report
70+ if: ${{ github.server_url != 'https://github.com' }}
71+ runs-on: ubuntu-latest
72+ timeout-minutes: 360
73+ env:
74+ DIFFINERY: ${{ github.server_url }}
75+ TOKEN: ${{ secrets.DIFFINERY_LIVE_TOKEN }}
76+ GITHUB_TOKEN_FOR_IMPORT: ${{ secrets.IMPORT_GITHUB_TOKEN }}
77+ steps:
78+ - uses: actions/checkout@v4
79+ - name: Import and run
80+ env:
81+ REPOS: ${{ inputs.repos }}
82+ MINUTES: ${{ inputs.minutes }}
83+ run: |
84+ [ -n "$GITHUB_TOKEN_FOR_IMPORT" ] || echo "::warning::IMPORT_GITHUB_TOKEN is not set, so imports use GitHub's anonymous rate limit"
85+ # shellcheck disable=SC2086
86+ REPORT=report.md scripts/import-campaign.sh $REPOS
87+ cat report.md >> "$GITHUB_STEP_SUMMARY"
88+ - uses: actions/upload-artifact@v4
89+ if: always()
90+ with:
91+ name: import-campaign-report
92+ path: report.md
modified.github/workflows/live-test.yml+59−1View file
@@ -2,7 +2,7 @@
22
33 # End-to-end checks against the real diffinery.com after each deploy: email
44 # sending, web UI features (syntax highlighting, settings pages, view transitions), hosted Actions jobs in Cloudflare Containers (with Docker builds and services), the container registry, agent sandboxes,
5-# webhooks, scheduled workflows, Git LFS, and pull requests (test merge commits, pull_request_target and the merge
5+# webhooks, scheduled workflows, Git LFS, imports from GitHub, and pull requests (test merge commits, pull_request_target and the merge
66 # queue). Each run works in a throwaway private repository that it
77 # deletes at the end.
88 #
@@ -470,6 +470,63 @@
470470 git fetch -q "$remote" main && [ "$(git show FETCH_HEAD:app.txt | tr '\n' ' ')" = "A b C " ] || { echo "::error::main does not hold the merged change"; exit 1; }
471471 echo "pull request #$n merged through the queue as $sha"
472472
473+ - name: Import a public GitHub repository
474+ if: steps.token.outputs.skip != 'true'
475+ env:
476+ # A small public repository with issues and pull requests; a repository variable can change it.
477+ SOURCE: ${{ vars.LIVE_IMPORT_SOURCE || 'octocat/git-consortium' }}
478+ run: |
479+ set -euo pipefail
480+ api() { curl -fsS -X "$1" "$DIFFINERY/api/v3$2" -H "Authorization: Bearer $TOKEN" -H "content-type: application/json" ${3:+--data "$3"}; }
481+ name="$REPO-import"
482+ # Token-free, so GitHub allows 60 requests an hour per address: a rate limit skips this check with a warning.
483+ code=$(curl -sS -o import.json -w '%{http_code}' -X POST "$DIFFINERY/api/v3/diffinery/imports" -H "Authorization: Bearer $TOKEN" -H "content-type: application/json" \
484+ --data "{\"source\":\"$SOURCE\",\"owner\":\"$OWNER\",\"name\":\"$name\",\"visibility\":\"private\"}")
485+ if [ "$code" = 429 ]; then echo "::warning::GitHub's rate limit stopped the import check: $(jq -r .message import.json)"; exit 0; fi
486+ [ "$code" = 201 ] || { echo "::error::starting the import returned $code: $(cat import.json)"; exit 1; }
487+ echo "IMPORT_REPO=$name" >> "$GITHUB_ENV"
488+ id=$(jq -r .id import.json)
489+ echo "import $id: $SOURCE -> $OWNER/$name"
490+ for i in $(seq 1 90); do
491+ imp=$(api GET "/diffinery/imports/$id")
492+ status=$(jq -r .status <<<"$imp")
493+ case "$status" in succeeded|failed|cancelled) break ;; esac
494+ if jq -e '[.steps[].message // ""] | any(test("rate limit"))' >/dev/null <<<"$imp"; then
495+ api POST "/diffinery/imports/$id/cancel" >/dev/null || true
496+ echo "::warning::GitHub's rate limit paused the import, so the import check is skipped: $(jq -r '[.steps[].message // ""] | map(select(test("rate limit")))[0]' <<<"$imp")"
497+ exit 0
498+ fi
499+ sleep 10
500+ done
501+ jq '{status, error, warnings, steps: [.steps[] | "\(.name): \(.status) \(.message // "")"], git}' <<<"$imp"
502+ [ "$status" = succeeded ] || { echo "::error::the import ended $status"; exit 1; }
503+
504+ # The same refs, issue and pull request numbers as on GitHub (when GitHub still answers this runner).
505+ R="/repos/$OWNER/$name"
506+ [ "$(api GET "$R" | jq -r .private)" = true ] || { echo "::error::the imported repository is not private"; exit 1; }
507+ gh() { curl -fsS "https://api.github.com/repos/$SOURCE$1" -H "Accept: application/vnd.github+json"; }
508+ if src=$(gh "") && branches=$(gh "/branches?per_page=100") && items=$(gh "/issues?state=all&per_page=30&sort=created&direction=asc"); then
509+ def=$(jq -r .default_branch <<<"$src")
510+ want=$(jq -r --arg b "$def" '.[] | select(.name == $b) | .commit.sha' <<<"$branches")
511+ got=$(api GET "$R/branches/$def" | jq -r .commit.sha)
512+ [ "$got" = "$want" ] || { echo "::error::$def is $got on Diffinery, $want on GitHub"; exit 1; }
513+ diff <(jq -r '.[].name' <<<"$branches" | sort) <(api GET "$R/branches?per_page=100" | jq -r '.[].name' | sort) || { echo "::error::the branches differ"; exit 1; }
514+ n=0
515+ for row in $(jq -r '.[] | "\(.number):\(if .pull_request then "pulls" else "issues" end)"' <<<"$items"); do
516+ num=${row%%:*} kind=${row#*:}
517+ want=$(jq -r --argjson n "$num" '.[] | select(.number == $n) | .title' <<<"$items")
518+ got=$(api GET "$R/$kind/$num" | jq -r .title)
519+ [ "$got" = "$want" ] || { echo "::error::$kind #$num is \"$got\" on Diffinery, \"$want\" on GitHub"; exit 1; }
520+ n=$((n + 1))
521+ done
522+ echo "$def at $got, $(jq length <<<"$branches") branches and $n issues and pull requests match GitHub"
523+ else
524+ echo "::warning::GitHub did not answer this runner (rate limit?), so the import is checked without comparing to GitHub"
525+ fi
526+ # No side effects: importing runs no workflows and sends no notifications.
527+ runs=$(api GET "$R/actions/runs" | jq .total_count)
528+ [ "$runs" = 0 ] || { echo "::error::the import started $runs workflow runs"; exit 1; }
529+
473530 - name: Scheduled workflow
474531 if: steps.token.outputs.skip != 'true'
475532 run: |
@@ -490,3 +547,4 @@
490547 curl -sS -o /dev/null -w 'package %{http_code}\n' -X DELETE "$DIFFINERY/api/v3/users/$OWNER/packages/container/${REPO,,}" -H "Authorization: Bearer $TOKEN"
491548 curl -sS -o /dev/null -w '%{http_code}\n' -X DELETE "$DIFFINERY/api/v3/repos/$OWNER/$REPO" -H "Authorization: Bearer $TOKEN"
492549 [ -z "${PR_REPO:-}" ] || curl -sS -o /dev/null -w '%{http_code}\n' -X DELETE "$DIFFINERY/api/v3/repos/$OWNER/$PR_REPO" -H "Authorization: Bearer $TOKEN"
550+ [ -z "${IMPORT_REPO:-}" ] || curl -sS -o /dev/null -w '%{http_code}\n' -X DELETE "$DIFFINERY/api/v3/repos/$OWNER/$IMPORT_REPO" -H "Authorization: Bearer $TOKEN"
addedscripts/import-campaign.sh+143−0View file
@@ -1,0 +1,143 @@
1+#!/usr/bin/env bash
2+# Imports real GitHub repositories into Diffinery as private test repositories, then
3+# runs their workflows there and reports what came across and what ran.
4+#
5+# Each source is imported as <owner>/campaign-<github owner>-<github repo>, replacing
6+# the copy from an earlier campaign. A commit to the default branch then starts the
7+# repository's push workflows, which are followed until they finish or MINUTES runs out.
8+# Jobs for runners Diffinery doesn't host (Windows, macOS) fail at once with a message saying so,
9+# which the report shows next to the job.
10+#
11+# Usage: DIFFINERY=https://diffinery.com TOKEN=... [GITHUB_TOKEN_FOR_IMPORT=...] \
12+# import-campaign.sh owner/repo [owner/repo...]
13+# Env: MINUTES (default 180) bounds the whole campaign; REPORT (default report.md).
14+set -euo pipefail
15+MINUTES=${MINUTES:-180}
16+REPORT=${REPORT:-report.md}
17+deadline=$(( $(date +%s) + MINUTES * 60 ))
18+api() { curl -fsS --retry 3 -X "$1" "$DIFFINERY/api/v3$2" -H "Authorization: Bearer $TOKEN" -H "content-type: application/json" ${3:+--data "$3"}; }
19+code() { curl -sS -o /tmp/campaign-out -w '%{http_code}' -X "$1" "$DIFFINERY/api/v3$2" -H "Authorization: Bearer $TOKEN" -H "content-type: application/json" ${3:+--data "$3"}; }
20+left() { echo $(( deadline - $(date +%s) )); }
21+
22+owner=$(api GET /user | jq -r .login)
23+declare -A import_id target
24+work=$(mktemp -d)
25+
26+# ------------------------------------------------------------ start every import
27+for src in "$@"; do
28+ name="campaign-${src//\//-}"
29+ name=${name,,}
30+ target[$src]=$name
31+ if [ "$(code GET "/repos/$owner/$name")" = 200 ]; then
32+ code DELETE "/repos/$owner/$name" >/dev/null
33+ fi
34+ body=$(jq -nc --arg s "$src" --arg o "$owner" --arg n "$name" --arg t "${GITHUB_TOKEN_FOR_IMPORT:-}" \
35+ '{source: $s, owner: $o, name: $n, visibility: "private"} + (if $t == "" then {} else {github_token: $t} end)')
36+ c=$(code POST /diffinery/imports "$body")
37+ if [ "$c" = 201 ]; then
38+ import_id[$src]=$(jq -r .id /tmp/campaign-out)
39+ echo "import $src -> $owner/$name (${import_id[$src]})"
40+ else
41+ echo "::error::$src: starting the import returned $c: $(jq -r '.message // .' /tmp/campaign-out)"
42+ jq -n --arg s "$src" --arg c "$c" --arg m "$(jq -r '.message // .' /tmp/campaign-out)" '{source: $s, status: "not started", error: "\($c): \($m)"}' > "$work/${name}.import.json"
43+ fi
44+done
45+
46+# ------------------------------------------------------------ wait for the imports
47+while :; do
48+ pending=0
49+ for src in "${!import_id[@]}"; do
50+ f="$work/${target[$src]}.import.json"
51+ [ -f "$f" ] && jq -e '.status | IN("succeeded", "failed", "cancelled")' "$f" >/dev/null && continue
52+ api GET "/diffinery/imports/${import_id[$src]}" > "$f" || { pending=$((pending + 1)); continue; }
53+ jq -e '.status | IN("succeeded", "failed", "cancelled")' "$f" >/dev/null || pending=$((pending + 1))
54+ done
55+ [ "$pending" = 0 ] && break
56+ if [ "$(left)" -le 0 ]; then echo "::warning::$pending imports were still running when the campaign ran out of time"; break; fi
57+ sleep 20
58+done
59+
60+# ------------------------------------------------------------ run each imported repository's push workflows
61+declare -A sha
62+for src in "${!import_id[@]}"; do
63+ name=${target[$src]}
64+ f="$work/$name.import.json"
65+ [ "$(jq -r .status "$f")" = succeeded ] || continue
66+ repo=$(api GET "/repos/$owner/$name")
67+ branch=$(jq -r .default_branch <<<"$repo")
68+ api GET "/repos/$owner/$name/actions/workflows" | jq '[.workflows[] | {name, path, state}]' > "$work/$name.workflows.json" || echo '[]' > "$work/$name.workflows.json"
69+ [ "$(jq length "$work/$name.workflows.json")" -gt 0 ] || continue
70+ put=$(jq -nc --arg b "$branch" --arg c "$(printf 'Imported from %s and run on Diffinery\n' "$src" | base64 -w0)" '{message: "Run the workflows on Diffinery", content: $c, branch: $b}')
71+ if out=$(api PUT "/repos/$owner/$name/contents/.diffinery-campaign" "$put"); then
72+ sha[$src]=$(jq -r .commit.sha <<<"$out")
73+ echo "$src: pushed ${sha[$src]} to $branch"
74+ else
75+ echo "::warning::$src: could not push a commit to $branch"
76+ fi
77+done
78+
79+finished() { jq -e '(.workflow_runs | length) > 0 and all(.workflow_runs[]; .status == "completed")' "$1" >/dev/null; }
80+# Runs can start other runs (workflow_run), so a set counts as finished after three quiet checks.
81+declare -A quiet
82+while :; do
83+ pending=0
84+ for src in "${!sha[@]}"; do
85+ name=${target[$src]}
86+ f="$work/$name.runs.json"
87+ [ "${quiet[$src]:-0}" -ge 3 ] && continue
88+ api GET "/repos/$owner/$name/actions/runs?head_sha=${sha[$src]}&per_page=100" > "$f" || { pending=$((pending + 1)); continue; }
89+ if finished "$f"; then quiet[$src]=$(( ${quiet[$src]:-0} + 1 )); else quiet[$src]=0; fi
90+ [ "${quiet[$src]}" -ge 3 ] || pending=$((pending + 1))
91+ done
92+ [ "$pending" = 0 ] && break
93+ [ "$(left)" -le 0 ] && break
94+ sleep 20
95+done
96+
97+# ------------------------------------------------------------ report
98+{
99+ echo "# Import campaign"
100+ echo
101+ echo "| Repository | Import | Took | Issues, PRs and comments | Warnings | Workflows | Runs |"
102+ echo "|---|---|---|---|---|---|---|"
103+} > "$REPORT"
104+details=""
105+for src in "$@"; do
106+ name=${target[$src]}
107+ f="$work/$name.import.json"
108+ status=$(jq -r .status "$f" 2>/dev/null || echo "?")
109+ took=$(jq -r 'if .started_at and .finished_at then (((.finished_at | sub("\\.[0-9]+Z$"; "Z") | fromdateiso8601) - (.started_at | sub("\\.[0-9]+Z$"; "Z") | fromdateiso8601)) / 60 | floor | "\(.) min") else "" end' "$f" 2>/dev/null || true)
110+ items=$(jq -r '[.steps[]? | select(.name == "issues" or .name == "comments") | "\(.count // 0) \(.name)"] | join(", ")' "$f" 2>/dev/null || true)
111+ warnings=$(jq -r '.warnings | length' "$f" 2>/dev/null || echo 0)
112+ workflows=$(jq -r 'length' "$work/$name.workflows.json" 2>/dev/null || echo 0)
113+ runs="-"
114+ if [ -f "$work/$name.runs.json" ]; then
115+ runs=$(jq -r '[.workflow_runs | group_by(.conclusion // .status)[] | "\(length) \(.[0].conclusion // .[0].status)"] | join(", ")' "$work/$name.runs.json")
116+ elif [ -n "${sha[$src]:-}" ]; then
117+ runs="none started"
118+ fi
119+ echo "| [$src](https://github.com/$src) → $owner/$name | $status | $took | $items | $warnings | $workflows | $runs |" >> "$REPORT"
120+
121+ details+=$'\n'"## $src"$'\n\n'
122+ [ "$status" = succeeded ] || details+="Import $status: $(jq -r '.error // "no error recorded"' "$f" 2>/dev/null)"$'\n\n'
123+ details+="$(jq -r '.steps[]? | "- \(.title // .name): \(.status)\(if .message then " (\(.message))" else "" end)"' "$f" 2>/dev/null)"$'\n'
124+ if [ "$warnings" != 0 ]; then details+=$'\n'"Warnings:"$'\n'"$(jq -r '.warnings[] | "- \(.)"' "$f")"$'\n'; fi
125+ [ -f "$work/$name.runs.json" ] || continue
126+ for run in $(jq -r '.workflow_runs[].id' "$work/$name.runs.json"); do
127+ r=$(jq -c --argjson id "$run" '.workflow_runs[] | select(.id == $id)' "$work/$name.runs.json")
128+ details+=$'\n'"### $(jq -r '"\(.name) [\(.event)]: \(.conclusion // .status)"' <<<"$r")"$'\n'
129+ jobs=$(api GET "/repos/$owner/$name/actions/runs/$run/jobs?per_page=100" || echo '{"jobs":[]}')
130+ details+="$(jq -r '.jobs[] | "- \(.name): \(.conclusion // .status)\(if .failure then " (\(.failure.message? // .failure))" elif .status != "completed" then " (still \(.status))" else "" end)"' <<<"$jobs")"$'\n'
131+ for job in $(jq -r '.jobs[] | select(.conclusion == "failure") | .id' <<<"$jobs"); do
132+ details+=$'\n'"<details><summary>$(jq -r --argjson id "$job" '.jobs[] | select(.id == $id) | .name' <<<"$jobs") log (last 40 lines)</summary>"$'\n\n```\n'
133+ details+="$(curl -fsS "$DIFFINERY/api/v3/repos/$owner/$name/actions/jobs/$job/logs" -H "Authorization: Bearer $TOKEN" | tail -40 || echo "(no log)")"
134+ details+=$'\n```\n</details>\n'
135+ done
136+ done
137+ # Runs still waiting (usually for a Windows or macOS runner) are cancelled so they don't sit in the queue.
138+ for run in $(jq -r '.workflow_runs[] | select(.status != "completed") | .id' "$work/$name.runs.json"); do
139+ code POST "/repos/$owner/$name/actions/runs/$run/cancel" >/dev/null || true
140+ done
141+done
142+echo "$details" >> "$REPORT"
143+cat "$REPORT"