Diffinery

codician/diffinerypublic

Open-source, agent-first GitHub and GitHub Actions alternative that runs on your own Cloudflare account: git hosting, pull requests, code review and drop-in GitHub Actions YAML on Workers, Durable Objects, R2 and Containers.

Docs: how web-only egress shows up in a job, and out-of-memory pushes (#141)

EGRESS=web now has a section on how a blocked port and an untrusted
certificate look in a job, a workflow to check it, and two runner
troubleshooting rows. Sandboxes, terminals, tasks and the architecture
page say where web-only applies. Git troubleshooting covers the
out-of-memory unpack message from #136.


Claude-Session: https://claude.ai/code/session_016q5wwBFck8JHq14gHUM2CL

Co-authored-by: Claude <noreply@anthropic.com>
Canberk Demirci authored · GitHub committed signed1 parent 2eb67abcommit e15c34e2a59b1204621f283eb3b8757dce9aabc5Browse files
7 files changed+39−8
Files (7)
modifiedapps/diffinery/docs/actions/runners.md+2−0View file
@@ -398,3 +398,5 @@
398398 | `sudo: command not found` on a hosted runner | Jobs run as root; remove `sudo`. |
399399 | `E: Unable to locate package ...` | Run `apt-get update` first; the image ships without package lists. |
400400 | `The runner stopped unexpectedly (exit code N).` | The hosted container exited, often from running out of memory. Reduce parallelism in the job (for example `make -j2`) or move it to a self-hosted runner. |
401+| `Connection refused` or `Connection timed out` on SSH (`git@...`), SMTP or a database port, on a hosted runner | The instance sets [`EGRESS=web`](/docs/self-hosting/configuration#egress), which lets hosted jobs use only HTTP and HTTPS. Use an `https://` remote or API, or run the job on a self-hosted runner. diffinery.com doesn't set it. |
402+| `unable to get local issuer certificate`, `x509: certificate signed by unknown authority` or another certificate error on a hosted runner | Under [`EGRESS=web`](/docs/self-hosting/configuration#how-a-blocked-connection-looks) HTTPS is re-signed with a Cloudflare CA. Most tools trust it already; a tool with its own CA list, or a Docker container the job starts, needs `/etc/cloudflare/certs/cloudflare-containers-ca.crt` added. |
modifiedapps/diffinery/docs/agents/sandboxes.md+2−2View file
@@ -7,7 +7,7 @@
77
88 ## What a sandbox is
99
10-- **A container per sandbox.** Each sandbox is its own Cloudflare Container, isolated from other sandboxes and from workflow jobs. It has outbound internet access.
10+- **A container per sandbox.** Each sandbox is its own Cloudflare Container, isolated from other sandboxes and from workflow jobs. It has outbound internet access (only HTTP and HTTPS on instances that set [`EGRESS=web`](/docs/self-hosting/configuration#egress)).
1111 - **The repository, checked out.** On start, the sandbox clones the repository and checks out the ref you asked for (the default branch if you don't say) into `/workspace/repo`.
1212 - **The hosted runner image.** Sandboxes run the same Ubuntu 24.04 image as hosted workflow jobs, so the same toolchains are there. See [What's inside](#whats-inside).
1313 - **An HTTP API, not a shell session.** Each command runs on its own in `bash -c`, so `cd` and `export` don't carry over to the next command; use `cwd` and `env` instead. Files stay between commands.
@@ -45,7 +45,7 @@
4545 | Utilities | `curl`, `jq`, `zip`, `unzip`, `xz-utils`, `tar`, `gzip`, `gnupg`, `openssh-client`, `ca-certificates` |
4646 | Docker | Docker Engine with the buildx and compose plugins; the daemon starts the first time `docker` runs |
4747
48-Install anything else with `apt-get`, `npm`, `pip` and so on; the sandbox has internet access.
48+Install anything else with `apt-get`, `npm`, `pip` and so on; the sandbox has internet access (only HTTP and HTTPS on instances that set [`EGRESS=web`](/docs/self-hosting/configuration#egress)).
4949
5050 Environment variables available to commands:
5151
modifiedapps/diffinery/docs/agents/tasks.md+1−1View file
@@ -148,7 +148,7 @@
148148
149149 Everything the sandbox returns, including the transcript, masks the model token and the git credential.
150150
151-**What this doesn't protect against.** Anything the agent runs in its sandbox (build scripts, tests, dependencies, code from the repository) can read the model token and the git credential while the run lasts, and repository content can try to trick the agent into misusing them ("prompt injection"). Sandboxes have internet access. So:
151+**What this doesn't protect against.** Anything the agent runs in its sandbox (build scripts, tests, dependencies, code from the repository) can read the model token and the git credential while the run lasts, and repository content can try to trick the agent into misusing them ("prompt injection"). Sandboxes have internet access (only HTTP and HTTPS on instances that set [`EGRESS=web`](/docs/self-hosting/configuration#egress)). So:
152152
153153 - Use a **dedicated API key with a spending limit** for Diffinery, and revoke it with the provider if you suspect misuse.
154154 - Run tasks on repositories whose content you trust.
modifiedapps/diffinery/docs/agents/terminal.md+1−1View file
@@ -45,7 +45,7 @@
4545
4646 - Git in the sandbox acts as the agent you picked, at most with write access to this one repository. The credential stops working when the terminal ends, and it's checked again every minute: if you or the agent lose write access, the terminal ends.
4747 - Your provider sign-in and anything else you type into the shell can be read by any process in the sandbox, including the sandbox's own server and its credentials, which reach only this sandbox. Don't paste secrets you wouldn't keep on that machine.
48-- The sandbox has outbound internet access, as other sandboxes do.
48+- The sandbox has outbound internet access, as other sandboxes do (only HTTP and HTTPS on instances that set [`EGRESS=web`](/docs/self-hosting/configuration#egress)).
4949
5050 ## When it ends
5151
modifiedapps/diffinery/docs/repositories/git.md+1−0View file
@@ -336,6 +336,7 @@
336336 | Git asks for a username and password when you push to a public repository you cloned without credentials | Expected: anonymous pushes get `401`, so Git asks for credentials. | Enter any username and your token as the password, or set up a credential helper with `diffinery auth setup-git`. |
337337 | `The requested URL returned error: 403` on push from a workflow | The job's token has no `contents: write` (`Resource not accessible by integration`). | Add `permissions: contents: write` to the workflow or job, or have an admin change the repository's default workflow permissions. |
338338 | `[remote rejected] ... (refusing to allow a GitHub App to create or update workflow ...)` | A workflow's job token tried to push a change to a workflow file. | Push workflow changes with a personal or agent token instead. |
339+| `remote unpack failed: the push needs more memory than one push may use; split it into smaller pushes (...)` | Unpacking or storing this push ran out of the memory one push may use, even though it is under the size limits. Nothing from the push is kept. | Push history in steps, oldest first, as for the 100 MB limit below. The GitHub importer does this by itself. |
339340 | `remote unpack failed: push exceeds 100 MB; split it into smaller pushes` | The pack for this push is larger than 100 MB. | Push history in steps, oldest first, for example `git push origin <older-commit>:refs/heads/main`, then later commits. Move large binaries to Git LFS. |
340341 | `remote: Push refused: tree <id>: duplicate entry '<name>'` (or another reason), `remote unpack failed: fsck error: ...` and `[remote rejected] ... (fsck failed)` | A tree or `.gitmodules` file in the push fails git's `fsck` checks: an entry that is empty, `.`, `..`, a spelling of `.git`, contains `/` or appears twice, a `.gitmodules` that is a symbolic link, or a submodule whose name has `..` or whose `url` or `path` starts with `-`. Nothing from the push is kept. | Run `git fsck` locally to find the object, fix the tree or `.gitmodules` in a new commit (or rewrite the commit that added it), and push again. |
341342 | `[remote rejected] ... (protected branch: required check '<name>' has not run on <sha>; ...)` | The branch requires status checks, and the commit you pushed hasn't passed them. | Push to another branch and open a pull request, or push a commit that already passed the checks. |
modifiedapps/diffinery/docs/self-hosting/architecture.md+2−2View file
@@ -163,8 +163,8 @@
163163
164164 All three container classes use the same image, built from `containers/runner/Dockerfile` (Ubuntu 24.04 with Node.js 22, git, git-lfs, Python and build tools, plus the runner agent).
165165
166-- **`RunnerContainer`** runs one hosted job (`runs-on: ubuntu-latest` and other `ubuntu-*` or `linux` labels). The Durable Object starts a container named after the job with three environment variables: `DIFFINERY_API_URL`, `DIFFINERY_JOB_ID` and `DIFFINERY_JOB_TOKEN`. The agent fetches its job spec from `/runner/jobs/<id>`, runs the steps, streams logs, uses the cache and artifact services and the OIDC endpoint, reports completion and exits. If the container stops before the job reports completion, the job fails with "The runner stopped unexpectedly". Instance type `standard-2`, at most 100 at once, internet access on. Docker is started inside the container the first time a job uses it.
167-- **`SandboxContainer`** runs one agent sandbox: the repository checked out at a ref, with an HTTP API on port 8080 for `exec`, file reads and file writes. The Worker forwards REST and MCP sandbox calls to it with a per-sandbox secret. Instance type `standard-1`, at most 50 at once, stopped after 30 minutes without requests.
166+- **`RunnerContainer`** runs one hosted job (`runs-on: ubuntu-latest` and other `ubuntu-*` or `linux` labels). The Durable Object starts a container named after the job with three environment variables: `DIFFINERY_API_URL`, `DIFFINERY_JOB_ID` and `DIFFINERY_JOB_TOKEN`. The agent fetches its job spec from `/runner/jobs/<id>`, runs the steps, streams logs, uses the cache and artifact services and the OIDC endpoint, reports completion and exits. If the container stops before the job reports completion, the job fails with "The runner stopped unexpectedly". Instance type `standard-2`, at most 100 at once, internet access on (only HTTP and HTTPS, through the Worker, under [`EGRESS=web`](/docs/self-hosting/configuration#egress)). Docker is started inside the container the first time a job uses it.
167+- **`SandboxContainer`** runs one agent sandbox: the repository checked out at a ref, with an HTTP API on port 8080 for `exec`, file reads and file writes. The Worker forwards REST and MCP sandbox calls to it with a per-sandbox secret. Instance type `standard-1`, at most 50 at once, stopped after 30 minutes without requests. Internet access is on, limited to HTTP and HTTPS under `EGRESS=web` like `RunnerContainer`, and a limited sandbox stays limited when it starts again after its idle stop.
168168 - **`ImportContainer`** runs the Git transfer of one GitHub import: the runner agent in import mode, started with `DIFFINERY_API_URL`, `DIFFINERY_IMPORT_ID` and `DIFFINERY_IMPORT_TOKEN`. One container per try, named `<import id>-<attempt>-<try>`. If it stops before the agent reports a result (a crash, running out of memory, a platform restart), the transfer counts as failed and the import's Workflow can try it again. Instance type `standard-4` (a mirror clone of a large repository needs disk and memory), at most 20 at once, internet access on, stopped after 7 hours without activity.
169169
170170 Self-hosted runners (`runs-on: [self-hosted, ...]`) run the same agent outside Cloudflare. They register with a repository admin's one-hour registration token, poll `POST /api/v3/repos/{owner}/{repo}/actions/runners/claim` with their own runner token (kept in the repository's Durable Object, hashed), and then talk to the same runner API. See [Runners](/docs/actions/runners).
modifiedapps/diffinery/docs/self-hosting/configuration.md+30−2View file
@@ -77,7 +77,7 @@
7777
7878 ### EGRESS
7979
80-**Experimental:** `web` hasn't been tried on a live instance yet, so turn it on for a test account first and run the checks below.
80+**Experimental:** `web` hasn't been tried on a live instance yet. The setting covers the whole instance, so try it on a test instance first, or check it with an account no site administrator runs, as in [Check it](#check-it) below. diffinery.com doesn't set it.
8181
8282 With `EGRESS=web`, hosted job and sandbox containers start without direct internet access. Cloudflare then lets only ports 80 and 443, and DNS to Cloudflare's resolvers, leave the container, and Diffinery forwards HTTP and HTTPS requests to any host through the Worker. Unset, or any other value, hosted compute reaches the whole internet as before. The setting is read when a container starts: jobs and sandboxes already running keep the access they started with.
8383
@@ -89,8 +89,36 @@
8989
9090 The account that owns the repository decides, as for [quotas](/docs/reference/limits#account-quotas). Accounts run by a site administrator (their own account, and organizations with a site administrator as an owner) stay open even with `EGRESS=web`, so the instance's own Deploy and live test workflows keep their access. Self-hosted runners and GitHub imports are not affected. Under `wrangler dev` sandboxes run in a local process, not containers, so the setting changes nothing there.
9191
92-To turn it on, add `"EGRESS": "web"` under `vars` in `wrangler.jsonc`; the deploy keeps it.
92+To turn it on, add `"EGRESS": "web"` under `vars` in `wrangler.jsonc`; the deploy keeps it. To turn it off, remove it (or set any other value) and deploy again; containers started after that are open.
9393
94+#### How a blocked connection looks
95+
96+Diffinery doesn't add a line to the job or terminal output when it limits a connection. The tool that made it reports an ordinary network error:
97+
98+- **A port other than 80 or 443** fails to connect, as behind a firewall: for example `ssh: connect to host github.com port 22: Connection refused` (or `Connection timed out`), or a mail or database client that can't reach its server.
99+- **HTTPS from a tool that doesn't trust the Cloudflare CA** fails its certificate check: for example `SSL certificate problem: unable to get local issuer certificate` (curl), `x509: certificate signed by unknown authority` (Go), `PKIX path building failed` (Java), or `self-signed certificate in certificate chain` (Node.js with its own CA list). Inside a Docker container a job starts, add the CA from `/etc/cloudflare/certs/cloudflare-containers-ca.crt` to the image, or run that step on the host.
100+
101+If the CA never appears in the container, the runner agent writes `web-only egress, but /etc/cloudflare/certs/cloudflare-containers-ca.crt did not appear: HTTPS will fail` to the container's own log (not the job log), and every HTTPS request from the job fails its certificate check.
102+
103+#### Check it
104+
105+With `EGRESS=web` deployed, run a workflow in a repository owned by an account that no site administrator runs (a site administrator's own repositories stay open):
106+
107+```yaml
108+on: workflow_dispatch
109+jobs:
110+ egress:
111+ runs-on: ubuntu-latest
112+ steps:
113+ - run: curl -fsSI https://example.com # works
114+ - run: git ls-remote https://github.com/git/git HEAD # works
115+ - run: npm view left-pad version # works
116+ - shell: bash
117+ run: "! timeout 10 bash -c '</dev/tcp/github.com/22'" # SSH port: must not connect
118+```
119+
120+All four steps pass. The last one fails if port 22 is reachable, which means the account is open (a site administrator runs it) or the setting isn't deployed.
121+
94122 ### DEV_AUTH
95123
96124 When `DEV_AUTH` is set, `ACCESS_AUD` is empty and both `PUBLIC_URL` and the request's own address are on the local machine (`localhost`, `127.x.x.x` or `[::1]`), every request without credentials (no token, no session) acts as the account with the email in `DEV_AUTH`, with full access. That account is created on first use and is a site administrator if the email is also in `BOOTSTRAP_ADMINS`. While `DEV_AUTH` is set, the email log also keeps each whole message, and `GET /api/v3/diffinery/emails` returns it, so tests can read emails without delivering them.