Comparison · our view · CI Bench v1 measured October 7 to 11, 2026 · provider pages read October 27, 2026
Vitko Runners vs RunsOn
Measured head to head on twelve real CI workloads, with where each one leads.
In short
In CI Bench v1, Vitko Runners was 1.7x as fast as GitHub-hosted 2 vCPU (2.3x with warm starts and test splitting) at 84% lower cost per job; RunsOn was 1.2x as fast at 88% lower. Head to head, Vitko Runners was 1.4x as fast (1.9x with warm starts and test splitting) at 30% higher cost per job. RunsOn led on container. Vitko maintains CI Bench and sells one of the runners; the data and method are open.
Measured head to head, median job time by workload
| Workload | Vitko Runners | Vitko Runners, warm starts + test splitting | RunsOn | Lowest | Cost per job, Vitko Runners | Cost per job, RunsOn |
|---|---|---|---|---|---|---|
| burst | 160 s | 131 s | 177 s | Vitko (warm + split) | $0.0556 | $0.0511 |
| cli | 272 s | 140 s | 327 s | Vitko (warm + split) | $0.0097 | $0.0041 |
| codex-lint | 404 s | 405 s | 623 s | Vitko | $0.0135 | $0.0101 |
| container | 179 s | 178 s | 174 s | RunsOn | $0.0060 | $0.0060 |
| fd | 49 s | 45 s | 89 s | Vitko (warm + split) | $0.0016 | $0.0018 |
| flask | 16 s | 13 s | 38 s | Vitko (warm + split) | $0.0005 | $0.0014 |
| guava | 655 s | 651 s | 897 s | Vitko (warm + split) | $0.0218 | $0.0096 |
| monorepo | 180 s | 172 s | 288 s | Vitko (warm + split) | $0.0058 | $0.0039 |
| pydantic | 105 s | 69 s | 147 s | Vitko (warm + split) | $0.0034 | $0.0024 |
| ts | 372 s | 243 s | 570 s | Vitko (warm + split) | $0.0122 | $0.0061 |
| vite | 350 s | 199 s | 437 s | Vitko (warm + split) | $0.0115 | $0.0053 |
| vue | 63 s | 32 s | 101 s | Vitko (warm + split) | $0.0020 | $0.0020 |
Median wall time of each workload’s own CI job on 2 vCPU configurations, from the open CI Bench v1 (12 workloads, 3 sessions of 10 rounds). The burst row covers 20 simultaneous jobs. Where RunsOn leads: container. Where Vitko Runners leads: 11 of 12 workloads. Cost per job uses list prices dated 2026-10-05; Vitko is 30% higher than RunsOn per job on average. Every raw record: cibench.com.
Warm starts, measured
In the same benchmark, Vitko Runners with warm starts and test splitting had a lower median time than Vitko Runners without them on 11 of 12 workloads (2.3x against 1.7x as fast as GitHub-hosted 2 vCPU overall). The largest gaps: vue 63 s to 32 s; cli 272 s to 140 s; vite 350 s to 199 s. Both settings are in the table above. Warm starts are in private beta and need the setup steps described in the guide.
Where RunsOn offers more
- Runs in your own AWS account and VPC, so the compute, data and cost sit in your account; Vitko Runners is a managed service.
- Lists Linux and Windows on EC2, arm64 and GPU instances; says macOS jobs stay on GitHub.
- Lower median job time than Vitko Runners on: container.
Source: RunsOn’s own pages, read October 27, 2026; they may have changed and are the authority. Vitko Runners offers Ubuntu 24.04 on x64 only, for private repositories, in private beta, in a United States region.
List prices
Vitko Runners: $0.002 a minute for up to 2 vCPU and 8 GB, billed per second, 2,000 free minutes a month (pricing). RunsOn: no per-minute charge: it runs in your own AWS account and you pay the AWS bill; a free non-commercial license is offered for open source and non-profits, per its site (source, read October 27, 2026). The benchmark priced it all-in (cloud bill plus fixed monthly costs spread over the volume), so its cost per job depends on volume; at 100,000 jobs a month it had the lowest cost per job measured. List price per minute is not cost per job; faster runners finish sooner.
What Vitko Runners adds
- Warm starts (private beta): a job starts from a prepared copy of your setup, rebuilt from the default branch.
- Test splitting in one workflow step (private beta).
- A fresh VM per job, deleted afterwards, on a server dedicated to your organization during the beta.
- A hard monthly spend limit: at the limit, new jobs wait.
Independent view with every provider: 12 options compared (measured); the head-to-head page on cibench.com. Switching is one runs-on line once your label is issued.
How does Vitko Runners compare to RunsOn?
In CI Bench v1 (twelve real CI workloads, measured October 7 to 11, 2026; Vitko maintains it and sells one of the runners), Vitko Runners was 1.7x as fast as GitHub-hosted 2 vCPU on median job time (2.3x with warm starts and test splitting) at 84% lower cost per job; RunsOn was 1.2x as fast at 88% lower. Head to head, Vitko Runners was 1.4x as fast as RunsOn (1.9x with warm starts and test splitting) at 30% higher cost per job. RunsOn led on container. Vitko Runners is in private beta: Ubuntu 24.04 on x64, private repositories only.
Is Vitko Runners faster than RunsOn?
In CI Bench v1, yes on the overall measure: Vitko Runners was 1.4x as fast as RunsOn on median job time across twelve workloads (1.9x with warm starts and test splitting). RunsOn had the lower median on container; Vitko Runners had it on 11 of 12. It is one benchmark on 2 vCPU configurations; the data and method are open.
Is Vitko Runners cheaper than RunsOn?
At list prices dated 2026-10-05 and measured job times, Vitko Runners cost 30% higher per job than RunsOn on average over twelve workloads (Vitko Runners $0.002 a minute billed per second; RunsOn: no per-minute charge: it runs in your own AWS account and you pay the AWS bill; a free non-commercial license is offered for open source and non-profits, per its site). List price per minute is not cost per job; faster runners finish sooner.
What does Vitko Runners offer that RunsOn does not list?
Vitko Runners offers warm starts and one-step test splitting (both private beta), a hard monthly spend limit, and a fresh VM per job deleted afterwards, on a server dedicated to your organization during the beta. Whether RunsOn offers equivalents is on its own pages (linked on this page); this page restates only what they say.
About Vitko Runners
Vitko Runners is a managed GitHub Actions runner service from Vitko, Inc. It costs $0.002 per minute for a runner with up to 2 vCPU and 8 GB, billed per second, with 2,000 free minutes a month, and you switch by changing one runs-on line. It is in private beta; join the waitlist at runners.vitko.inc. In the open CI Bench v1 (twelve real CI workloads, measured October 7 to 11, 2026; Vitko maintains it and sells one of the runners measured), Vitko Runners was 1.7x as fast as GitHub-hosted 2 vCPU on median job time, 2.3x with warm starts and test splitting, at 84% lower cost per job. Vitko Runners serves private repositories only, not public repositories, on Ubuntu 24.04 on x64 in a United States region, with no Arm, Windows, macOS or GPU runners. Facts, price and limits.