Skip to content

Inside the Skill Market: From Software Engineering Activities to Reusable Agent Skills

🕒 Published (v1): 2026-07-10 03:21 UTC · Source: Arxiv · link

Why this paper was selected

Jialun Cao (HKUST); formalizes reusable agent skills from SE activities — modular agent composition architecture

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

This paper presents the first large-scale empirical study of SE-related agent skills in public skill marketplaces, analyzing 11,497 unique skills collected from four platforms (ClawHub, SkillHub, SkillNet, SkillsMP). It characterizes how software engineering activities are being encapsulated into reusable agent skill artifacts, revealing uneven lifecycle coverage and weak evaluation mechanisms.

Problem

No systematic understanding exists of which software engineering activities are being packaged into reusable agent skills within emerging skill ecosystems. Prior work covers broad cross-domain skill analyses, safety, or benchmarking—but lacks an activity-centric view of SE-specific skills and their distribution across the software development lifecycle.

Method

An automated collection pipeline crawled four skill marketplaces using 110 SE-related keywords, gathering 775,790 raw skills. After intra- and cross-market deduplication, a two-stage relevance filter was applied: (1) rule-based lexicon matching against SE-positive and non-SE-negative term lists, and (2) LLM-based filtering using GPT-5.5 to conservatively classify each skill by name and description. The intersection of both filters, minus 2,067 deleted-repository skills, yielded 11,497 unique SE-related skills. These were analyzed across three research questions covering structural properties (RQ1), SE lifecycle coverage (RQ2), and evaluation practices (RQ3). Structural analysis distinguished descriptive components (Frontmatter, Usage, Commands, Examples, Workflow, Verification in SKILL.md) from functional asset forms (Instruction, Documentation, Script, Agent Workflow, Library, Application).

Key Contributions

  • First activity-centric empirical characterization of 11,497 SE-related agent skills drawn from four public marketplaces (data collected June 2026).
  • A structured mapping from SE lifecycle activities to skill artifacts, exposing which stages (e.g., coding, testing, security) are over-represented and which (e.g., requirements analysis, software design) are underrepresented.
  • Analysis of skill evaluation practices and ecosystem-level composition patterns, revealing a gap between current evaluation mechanisms and true reuse effectiveness.

Results

  • Scale and growth: 11,497 unique SE-related skills; skill update counts rose sharply after December 2025, reaching 2,042 updates in January 2026 and continuing to grow through May 2026.
  • Source distribution: SkillsMP is the largest contributor (5,322 skills, 46.3%); only 722 skills (6.3%) overlap across marketplaces, indicating ecosystem fragmentation.
  • Skill length: Average SKILL.md length is 2,078 tokens; 90th/95th/98th percentiles are 4,150 / 5,565 / 7,971 tokens; top 1% exceed 11,185 tokens.
  • Structural form: 63.8% of skills are pure Instruction (natural-language only); 18.6% Documentation; only 10.5% Script, 2.0% Library, 1.1% Application; 3.9% Agent Workflow—meaning only ~13.6% include any executable code.
  • Descriptive completeness: Frontmatter coverage 97.8%; Commands 79.3%; Verification 73.7%; Workflow 60.1%; Usage 51.3%; Examples 50.1%—nearly half of skills lack usage instructions or concrete examples.
  • Versioning: Most skills have very few versions; skills with >20 versions exist but manual inspection shows many reflect repeated releases without substantive changes rather than genuine evolution.
  • Marketplace focus: ClawHub centers on coding automation and Git workflows; SkillNet on software development and QA; SkillHub on developer workflows and design; SkillsMP on security analysis and configuration automation.

Limitations

  • Data collection completed June 2026 from four specific marketplaces only; coverage of the broader skill ecosystem (e.g., proprietary or enterprise repositories) is absent.
  • Relevance filtering intersects rule-based and LLM-based methods, but GPT-5.5 judgment on borderline cases may introduce noise; the paper does not report inter-rater reliability.
  • RQ2 (lifecycle coverage) and RQ3 (evaluation practices) results are only partially reported in the provided text excerpt; quantitative findings for those sections are not fully visible.
  • Skills with deleted repositories (2,067 removed) may bias the surviving dataset toward actively maintained skills.
  • The study is purely observational/empirical; it does not evaluate whether the skills, when executed by agents, actually achieve their stated SE tasks.

Relevance to Agentic AI / LLM Agents

This paper directly maps the emerging ecosystem of reusable agent skills—a core building block of agent-centric SE—providing the first principled audit of what capabilities are actually being packaged and shared. For researchers tracking agentic AI, it exposes a structural gap: the vast majority of SE skills are plain-text instructions rather than executable or compositional artifacts, which constrains true agent autonomy. The finding that high-context SE lifecycle stages (requirements, design) remain largely uncovered is a concrete gap for future work on skill engineering and agent planning. The study also connects to active lines of work on skill acquisition (Voyager-style), multi-agent workflows (MetaGPT, ChatDev), and skill safety, situating skill market analysis as a missing empirical lens.