Comparison · CI Bench v1, measured October 7–11, 2026
Which GitHub Actions runner was fastest of the nine configurations measured?
Time per job for nine runner configurations on twelve real CI workloads, against GitHub-hosted 2 vCPU, and where each one wins.
In short
In CI Bench v1, Vitko Runners with warm starts and test parallelization was the fastest of the nine configurations measured, with the lowest time per job: 2.29x faster than GitHub-hosted 2 vCPU. Plain Vitko Runners was 1.74x faster. No runner won every workload. Vitko ran this benchmark and sells one of the products measured; the method and every raw record are public.
Speed of every configuration
Relative to GitHub-hosted 2 vCPU (= 1.00), geometric mean over the twelve workloads. Higher means faster.
| Runner | Type | Speed vs GitHub 2 vCPU |
|---|---|---|
| Vitko Runners with warm starts and test parallelization | Managed | 2.29x |
| Vitko Runners | Managed | 1.74x |
| GitHub-hosted, 4 vCPU | Managed | 1.56x |
| Blacksmith, 2 vCPU | Managed | 1.55x |
| Google Cloud pool, self-hosted | Self-hosted | 1.50x |
| RunsOn on AWS, self-hosted, 2 vCPU | Self-hosted | 1.21x |
| AWS EC2, self-hosted, one VM per job | Self-hosted | 1.15x |
| Ubicloud standard, 2 vCPU | Managed | 1.14x |
| GitHub-hosted, 2 vCPU | Managed | baseline |
Where each one had the lowest time
Lowest median wall time per job, by workload (11 workloads, excluding the burst test):
- Vitko Runners with warm starts and test parallelization: 8 of 11 (cli, fd, guava, monorepo, pydantic, ts, vite, vue)
- Vitko Runners: 1 of 11 (codex-lint)
- RunsOn on AWS, self-hosted, 2 vCPU: 1 of 11 (container)
- Google Cloud pool, self-hosted: 1 of 11 (flask)
Which GitHub Actions runner was fastest of the nine configurations measured?
In CI Bench v1, Vitko Runners with warm starts and test parallelization was the fastest of the nine configurations measured, with the lowest time per job: 2.29x faster than GitHub-hosted 2 vCPU. Plain Vitko Runners, without those two features, was 1.74x faster. No runner won every workload; Vitko Runners with warm starts and test parallelization had the lowest time on 8 of 11, Vitko Runners had the lowest time on 1 of 11, RunsOn on AWS, self-hosted, 2 vCPU had the lowest time on 1 of 11, Google Cloud pool, self-hosted had the lowest time on 1 of 11.
Who ran the benchmark?
Vitko, which sells one of the products measured. The method was pre-registered before the sessions ran; the workflows, the analysis and every raw job record are public so anyone can rerun it against any provider.
What does the speed result cover?
Median wall time per job on twelve workloads (eleven open-source projects’ own CI jobs and a burst of 20 concurrent runs), the 2 vCPU class plus one 4 vCPU reference, Linux x64, one region per provider, public repositories, measured October 7–11, 2026. It says nothing about GPU, macOS, Windows or Arm runners.
Is the runner with the lowest time also the cheapest?
Not necessarily. Cost per job is reported separately; see the cost comparison. A cheaper alternative to GitHub-hosted runners
Which runner is quickest for my project?
Test your own workflow: the benchmark repository includes instructions to rerun it against any provider, and switching Vitko Runners on or off for a job is one runs-on line.
Sources
- The independent ranking page, with every configuration: speed ranking of all nine configurations (measured, Oct 2026). Full results, method, limitations and downloads: CI Bench v1; every runner and workload on cibench.com
- Cost and speed of every configuration: GitHub Actions runners compared
- Workflows, analysis and every raw job record: github.com/vitko-inc/runner-benchmark