Docker on hosted runners: start the daemon as a Docker job begins, again (#230)
* Docker on hosted runners: the daemon starts as a Docker job begins again
3 files changed+4−5
modifiedapps/diffinery/docs/actions/docker.md+1−1View file
| @@ -14,7 +14,7 @@ | ||
| 14 | 14 | |
| 15 | 15 | On hosted runners: |
| 16 | 16 | |
| 17 | -- The `docker` command starts the daemon on first use and prints `Starting the Docker daemon`. Jobs that never use Docker don't start it. A docker command that has waited over 2 minutes for another one to start the daemon fails the step with `The Docker daemon did not start:` instead of hanging. | |
| 17 | +- The `docker` command starts the daemon on first use and prints `Starting the Docker daemon`. Jobs that never use Docker don't start it. A job that will use Docker (a job `container:`, `services:`, a `docker://` action, or a step whose `uses:` or `run:` names `docker`) gets the daemon started in the background as the job begins, so it is usually ready by the time checkout and setup are done. A docker command that has waited over 2 minutes for another one to start the daemon fails the step with `The Docker daemon did not start:` instead of hanging. | |
| 18 | 18 | - Docker Hub images are pulled through the registry mirror `https://mirror.gcr.io`, which avoids Docker Hub's anonymous pull limits for official images. Images the mirror doesn't have come from Docker Hub directly. |
| 19 | 19 | - Containers use the DNS servers `1.1.1.1` and `8.8.8.8`. |
| 20 | 20 | - Bridge networks, service names, port publishing and image builds all work, as on GitHub's hosted runners. |
modifiedapps/diffinery/docs/actions/runners.md+1−1View file
| @@ -155,7 +155,7 @@ | ||
| 155 | 155 | |
| 156 | 156 | Docker works on hosted runners the same way as on GitHub's: job containers, service containers, Docker container actions, and `docker build`, `docker run` and `docker compose` in `run:` steps. Details: |
| 157 | 157 | |
| 158 | -- **Started on first use.** The `docker` command is a wrapper that starts the Docker daemon the first time anything in the job needs it (a `docker` command in a step, a job `container:`, `services:`, or a Docker action). The first use prints `Starting the Docker daemon` and takes a few seconds; jobs that never touch Docker don't pay for it. Docker commands that arrive while another one is starting the daemon wait for it. If the daemon can't start, or another command has been starting it for over 2 minutes, the step fails with `The Docker daemon did not start:` followed by the end of the daemon's log, rather than waiting until the job's timeout. | |
| 158 | +- **Started on first use.** The `docker` command is a wrapper that starts the Docker daemon the first time anything in the job needs it (a `docker` command in a step, a job `container:`, `services:`, or a Docker action). The first use prints `Starting the Docker daemon` and takes a few seconds; jobs that never touch Docker don't pay for it. A job that will use Docker (a job `container:`, `services:`, a `docker://` action, or a step whose `uses:` or `run:` names `docker`) gets the daemon started in the background as the job begins, so those seconds overlap checkout and setup. Docker commands that arrive while the daemon is starting wait for it. If the daemon can't start, or another command has been starting it for over 2 minutes, the step fails with `The Docker daemon did not start:` followed by the end of the daemon's log, rather than waiting until the job's timeout. | |
| 159 | 159 | - **Docker Hub through `mirror.gcr.io`.** The daemon uses `https://mirror.gcr.io` as its registry mirror, so pulls of Docker Hub images (`redis:7-alpine`, `postgres:16`) usually come from Google's public mirror and avoid Docker Hub's anonymous rate limits. Images the mirror doesn't have are pulled from Docker Hub as usual. |
| 160 | 160 | - **DNS.** The daemon gives containers the DNS servers `1.1.1.1` and `8.8.8.8`. |
| 161 | 161 | - **Pushing to the Diffinery registry.** The daemon and BuildKit builders reach `diffinery.com` through a local relay that splits image layer uploads into chunks, so layers larger than Cloudflare's 100 MB request limit push from hosted workflows. See [Container registry](/docs/packages/container-registry). |
modifiedcontainers/runner/src/main.ts+2−3View file
| @@ -150,9 +150,8 @@ | ||
| 150 | 150 | // two are for cancelling the steps and reporting the result before SIGKILL. |
| 151 | 151 | drainOnSigtermMs: args.spec ? undefined : 13 * 60_000, |
| 152 | 152 | // A hosted runner is its own machine (Cloudflare sets the Durable Object id in its container). |
| 153 | - // Off until the docker wrapper's lock fix (image/docker) is live and verified: a docker command | |
| 154 | - // that ran while the head start was still starting the daemon waited on the lock forever. | |
| 155 | - prewarmDocker: false, | |
| 153 | + // test/docker-wrapper.test.ts runs the image's docker wrapper with the head start on. | |
| 154 | + prewarmDocker: !!env.CLOUDFLARE_DURABLE_OBJECT_ID, | |
| 156 | 155 | measureUsage: !!env.CLOUDFLARE_DURABLE_OBJECT_ID || env.DIFFINERY_MEASURE_USAGE === "1" ? new UsageSampler({ cores: Number(env.DIFFINERY_RUNNER_VCPU) || undefined }) : false, |
| 157 | 156 | }); |
| 158 | 157 | process.stdout.write(`[diffinery-runner] job finished: ${completion.conclusion}\n`); |