🔄 rerun-to-green
rerun-to-green
How many failed CI runs go green on a bare rerun — no code change?
Measured on 143,061 public GitHub Actions runs from 32 open-source repositories (2026-08-19).
🧪 Real numbers
📐 Measured: 2026-08-19
🔄 Rerun only
⚖️ Honest limits
10,047
failed-run episodes
1,785
re-run on the same commit
498
turned green, zero code change
~28%
same-commit recovery (entry-level)
The question
Every developer knows the reflex: “CI is red. My diff looks fine. Let me just rerun it.”
How often does that actually work? We measured it on public data — no estimates.
The number
And the press-rerun case, validated on a random sample: ~75% of runs that were re-run on a failed first attempt finished green.
How we measured
- Downloaded workflow-run metadata for 32 public repos via the GitHub API.
- Grouped runs by (repository, workflow, commit SHA).
- Found episodes that started red (failure / timed_out).
- Checked whether a later run of the exact same commit finished green.
- Validated the press-rerun case on a random sample (attempt #1 fetched and confirmed as a genuine failure).
No source code. No logs. No user data. Public metadata only.
Results
| Metric | Value |
|---|---|
| Failed runs re-run on the same commit | ~18% |
| …of which turned green (entry-level) | ~28% |
| …of which turned green when you press rerun (attempt-level) | ~75% |
| Median time-to-green (recovered) | ~4 min |
| P90 time-to-green (recovered) | ~2.9 h |
What this does and does not mean
- Green after rerun ≠ code is correct. It means the failure wasn't caused by your commit in that run.
- This is not “AI causes this”. We don't know which runs were AI-written and we don't claim to.
- Selection bias: developers re-run runs they suspect are flaky, so ~75% is conditional on choosing to rerun.
- Snapshot: 2026, up to 90 days, 32 curated repos; heavy repos capped at 6,000 most recent runs.
The point
AI coding tools are measured by how fast a patch is written. But a patch isn't done when it's written — it's done when CI is green.
AI writes the patch. CI decides whether it's usable.
Reproduce
GH_TOKEN=<token> WINDOW_DAYS=90 python3 src/fetch_runs.py python3 src/analyze.py python3 src/validate_attempts.py