Skip to content

Early Adoption of Agentic Coding Tools by GitHub Projects

🕒 Published (v1): 2026-07-15 17:05 UTC · Source: Arxiv · Venue: KDD 2026 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

This paper presents a large-scale empirical analysis of 25,264 agentic PRs across 2,361 popular GitHub repositories (May–July 2025) to characterize how open-source projects adopt Copilot, Codex, and Claude Code. Adoption is widespread but shallow: the median repo produces only 1–2 agentic PRs in three months, and human oversight is almost universally a single-developer model.

Problem

Prior work on AI coding agents focuses on PR-level outcomes (acceptance, quality, labor allocation) rather than project-level adoption dynamics. It is unknown how adoption rates, agent productivity, and human oversight structures vary across projects of different team sizes, leaving a gap in understanding the organizational conditions that govern sustainable human-agent collaboration.

Method

The study mines the AIDev-pop dataset—GitHub repos with ≥100 stars active during the study period—filtered to merged and closed PRs generated by Copilot, Codex, or Claude Code from May–July 2025. Bot/agent commits are identified via keyword matching on committer fields (e.g., codex, claude, copilot, [bot]). Repositories are stratified into small (1–5 contributors), medium (6–15), and large (16+) teams via the GitHub /contributors API. Three metrics are computed: (1) human participation ratio (fraction of repo contributors active in any agentic PR), (2) agentic PRs per human participant (proxy for productivity), and (3) human-agent collaboration pattern (five mutually exclusive categories based on reviewer/committer count and identity). Statistical comparisons use Kruskal-Wallis H-tests with Dunn's post-hoc correction and Cliff's Delta effect sizes.

Key Contributions

  • First project-level (vs. PR-level) empirical characterization of agentic coding tool adoption across 2,361 open-source repositories.
  • Quantitative evidence that small projects (1–5 contributors) have dramatically higher human participation ratios in agentic workflows than medium or large projects (Cliff's \(\delta = 0.9969\) for small vs. large).
  • Measurement of agentic PR productivity against an industry benchmark of 36 PRs/developer/quarter; only 1% of projects (25/2,361) exceed this threshold.
  • Taxonomy of five human-agent collaboration patterns and their distribution: 78.9% of agentic PRs follow a single-human review-and-commit model.

Results

  • Adoption breadth: Median agentic PR count is 1–2 per repo across all project sizes during the three-month window; 42.3% of projects have a human participation ratio below 0.05.
  • Small-project skew: Small projects average 50.2 agentic PRs/repo vs. 5.6 (medium) and 6.7 (large), but this mean is heavily skewed by a few outliers.
  • Participation ratio by size: Kruskal-Wallis \(H = 1211.79\), \(p < 0.001\), \(\eta_H^2 = 0.6157\); all pairwise Cliff's \(\delta\) values are large (≥ 0.90), robust across alternative size cutoffs.
  • Productivity: Only 25 projects (1%) exceed the 36-PR/participant reference line; high productivity is concentrated in projects with very few human participants.
  • Collaboration patterns: "1 Reviewer + 1 Committer (Same Person)" accounts for 78.9% of PRs (19,488); adding "1 Reviewer, No Committer" (9.8%) brings single-human oversight to 88.7%; multi-human patterns cover only 11.3%.
  • Larger projects show somewhat more distributed review practices, but single-human workflows remain dominant at all team sizes.

Limitations

  • Observation window is only three months (May–July 2025), capturing a single early-adoption snapshot; longitudinal dynamics are unknown.
  • Agent identification relies on keyword matching, which may miss novel agents or misclassify accounts.
  • Contributor count from the GitHub API is a coarse proxy for team size and does not capture active vs. inactive contributors during the study period.
  • The 36 PRs/participant benchmark is borrowed from industry reports on human developers and may not be a valid productivity target for agentic workflows.
  • Agentic PRs are not normalized by size or complexity, so productivity is measured purely by volume.
  • Only merged and closed PRs are included; open PRs are excluded, potentially undercounting active adoption.
  • Dataset is limited to repositories with ≥100 stars, which may not generalize to less popular projects.

Relevance to Agentic AI / LLM Agents

This study provides rare ground-truth evidence on how Copilot, Codex, and Claude Code are actually deployed at scale in real engineering workflows, not in benchmarks. The dominance of single-human oversight (88.7% of PRs) quantifies the current human-in-the-loop norm and has direct implications for designing agentic systems that fit existing review capacity rather than assuming distributed team coordination. The finding that intensive adoption is concentrated in a small minority of projects suggests that agent capability alone is insufficient—organizational processes and contributor structures are co-determinants of adoption depth, a key consideration for agentic system designers and researchers studying human-agent teaming. The empirical gap between the median repo (1–2 PRs/quarter) and high-performing outliers also identifies a measurement baseline against which future agent improvements can be tracked.