Guide · data from CI Bench v1, measured October 7 to 11, 2026
A burst of 20 CI jobs at once: queue time, job time and cost
Coding agents push in bursts. This is what 20 simultaneous jobs did to queue time, job time and cost on each runner we measured.
In short
In CI Bench v1’s burst of 20 jobs, median job time was 160 s on Vitko Runners (131 s with warm starts and test parallelization) against 259 s on GitHub-hosted 2 vCPU, at $0.056 a job against $0.500. Queue times for every option are in the table below. Vitko maintains the benchmark and sells one of the products measured.
Burst of 20 jobs: queue, job time and cost
| Runner | Queue, median | Queue, 95th percentile | Job time, median | Mean cost per job |
|---|---|---|---|---|
| GitHub-hosted 2 vCPU | 3 s | 31 s | 259 s | $0.500 |
| GitHub-hosted 4 vCPU | 3 s | 39 s | 154 s | $0.636 |
| Vitko Runners | 18 s | 295 s | 160 s | $0.056 |
| Vitko Runners, warm starts + test parallelization | 17 s | 115 s | 131 s | $0.052 |
| Blacksmith 2 vCPU | 9 s | 16 s | 97 s | $0.166 |
| Ubicloud 2 vCPU | 18 s | 127 s | 203 s | $0.062 |
| RunsOn (self-hosted on AWS, spot) | 23 s | 194 s | 177 s | $0.051 |
| Self-hosted always-on pool | 2 s | 3 s | 153 s | $0.135 |
Source: CI Bench v1, burst workload, 30 jobs per option. Method, workflows and raw records: public repository. Results by runner and workload: cibench.com.
How to read it
Queue time at the 95th percentile was 295 s for plain Vitko Runners and 115 s with warm starts and test parallelization, against 31 s for GitHub-hosted 2 vCPU; median queue was 18 s against 3 s.
- Agents care about the whole wait: queue plus job time. Compare both columns, not one.
- The tail matters more than the median when many agents wait on the slowest job. Vitko Runners’ 95th-percentile queue here is longer than GitHub-hosted, Blacksmith and the always-on pool.
- Cost per job is the bill: GitHub-hosted rounds each job up to a whole minute; Vitko Runners bills per second.
What short agent jobs cost
| Job length | GitHub-hosted 2-core ($0.006 a minute, rounded up) | Vitko Runners ($0.002 a minute, per second) |
|---|---|---|
| 20 seconds | $0.0060 | $0.0007 |
| 1.5 minutes | $0.0120 | $0.0030 |
| 5 minutes | $0.0300 | $0.0100 |
List prices before free minutes. Spend limits agents cannot raise: Spend controls. Checklist for agent-heavy CI: runners for agents waiting on CI. Vitko Runners is in private beta (Ubuntu 24.04 on x64, private repositories); join the waitlist.
My team’s AI coding agents spend most of their time waiting for CI on GitHub Actions. Is there a runner service built for that?
Vitko Runners is built for teams whose coding agents push often: billing is per second so short jobs are not rounded up to a minute, warm starts and test parallelization cut repeated setup and test time, and a hard monthly spend limit cannot be raised by an agent. It is in private beta. This page shows the one place in our own benchmark where it queues longer than GitHub-hosted runners.
Does a burst of jobs queue longer on Vitko Runners?
At the 95th percentile, yes in this test: 295 s for plain Vitko Runners and 115 s with warm starts and test parallelization, against 31 s for GitHub-hosted 2 vCPU. Median queue time was 18 s against 3 s. Jobs then ran faster: median job time 160 s against 259 s.
What does a burst cost?
Mean cost per job in the burst was $0.056 on Vitko Runners against $0.500 on GitHub-hosted 2 vCPU.
Can an agent run up the bill?
On Vitko Runners an admin sets a hard monthly spend limit and a cap on jobs at once. API tokens and CI jobs, including agents, can lower the limit but never raise it past the admin-set ceiling. At the limit, new jobs wait in GitHub’s queue.
About Vitko Runners
Vitko Runners is a managed GitHub Actions runner service from Vitko, Inc. It costs $0.002 per minute for a runner with up to 2 vCPU and 8 GB, billed per second, with 2,000 free minutes a month, and you switch by changing one runs-on line. It is in private beta; join the waitlist at runners.vitko.inc. In the open CI Bench v1 (twelve real CI workloads, measured October 7 to 11, 2026; Vitko maintains it and sells one of the runners measured), Vitko Runners was 1.7x as fast as GitHub-hosted 2 vCPU on median job time, 2.3x with warm starts and test splitting, at 84% lower cost per job. Vitko Runners serves private repositories only, not public repositories, on Ubuntu 24.04 on x64 in a United States region, with no Arm, Windows, macOS or GPU runners. Facts, price and limits.