SkillReducer: Optimizing LLM Agent Skills for Token Efficiency
SkillReducer: Optimizing LLM Agent Skills for Token Efficiency — L2 #
§1 TL;DR #
SkillReducer is a two-stage skill debloating framework that compresses LLM agent skill descriptions by 48% and bodies by 39% via delta-debugging-based routing optimization and taxonomy-driven progressive disclosure, improving functional quality by 2.8% (less-is-more effect) with 0.965 cross-model retention.
§2 Q1 — 痛点 / Q2 — 方法 / Q3 — 结果 #
Q1 痛点 #
LLM coding agents (Claude Code, Cursor, Windsurf) rely on "skills" — pre-packaged instruction sets — that are injected into the context window upon selection. An empirical study of 55,315 public skills reveals three systemic inefficiencies:
- F1 — Missing/bloated descriptions: 26.4% of skills have no routing description at all; those that do often contain verbose non-routing content. Skills without descriptions still consume router token budget on every invocation but never match.
- F2 — Non-actionable body content: Only 38.5% of body content is actionable core rules; over 60% is background, examples, or templates injected regardless of task relevance.
- F3 — Reference overloading: 14.8% of skills include reference files that inject massive token volumes (1.67M tokens across 100 SkillHub skills) without filtering by task need.
At current API pricing, a 10,000-token skill costs \$0.03–\$0.15 per invocation, and teams with frequent agent use can spend hundreds of dollars monthly on skill content alone. This "skill bloat" — analogous to software bloat — inflates token costs while simultaneously degrading agent performance through attention dilution.
Agent scope: SkillReducer targets coding agents with skill-based capability extension. The interaction pattern is multi-turn (skills persist across conversation turns). The framework operates as a build-time preprocessing step — fully automated, no human-in-the-loop.
Q2 方法 #
SkillReducer is a two-stage debloating framework inspired by software debloating, mapping each empirical finding to a design requirement:
Stage 1 — Routing Layer Optimization (addresses F1):
- Segments descriptions into semantic clauses, then applies delta debugging (ddmin) with a simulated routing oracle containing adversarial distractors to find 1-minimal routing-sufficient descriptions.
- Two-phase design: Phase 1 uses a fast simulated oracle (~2s/call) for ddmin's $O(n \log n)$ iterations; Phase 2 validates against real agent runtime (Claude Code CLI) with selective restore of up to 3 deleted clauses if routing breaks.
- For the 26.4% with missing/short descriptions (≤40 tokens), generates from body by extracting three routing signals: primary capability, trigger condition, unique identifiers.
Stage 2 — Body Restructuring via Progressive Disclosure (addresses F2, F3):
- Classifies body content into 5 taxonomy categories (core rule / background / example / template / redundant) using an LLM classifier.
- Core rules remain always-loaded; background/examples/templates move to on-demand reference modules loadable via
read_filetool calls. - Type-specific compression: core rules merged and tightened; examples deduplicated (60–70% count reduction); templates deduplicated; background summarized.
- Cross-file deduplication removes overlap between references and body; surviving references annotated with "when" clauses and topic keywords.
- Two quality gates: Gate 1 (faithfulness — all operational concepts preserved) and Gate 2 (task-based — compressed skill achieves $\text{score}_C \geq \text{score}_A$ on 5 diverse tasks) with a feedback loop that promotes failed non-core items to core (up to 2 iterations).
核心技术壁垒: The taxonomy-driven classification is the single hardest-to-replicate insight. The ablation shows a 6.8pp retention gap when classification is removed (C4 retention 0.919 vs. C full pipeline 0.987). Flat compression — even with the same token budget — actively harms skill quality because it cannot distinguish operationally critical rules from illustrative examples. Building a reliable classifier that separates "actionable" from "illustrative" in natural-language instructions requires both the taxonomy design (validated by GMM clustering, silhouette 0.393) and the conservative fallback (classifier failure defaults to core_rule, never accidentally evicting critical content).
Q3 结果 #
Evaluated on 600 skills and the SkillsBench benchmark:
| Metric | Value |
|---|---|
| Description compression | 48% mean (median 59%) |
| Body compression | 39% mean (median 43%) |
| Pass rate ($\text{score}_C \geq \text{score}_A$) | 86.0% (95% CI: [83.1%, 88.7%]) |
| Improvement rate ($\text{score}_C > \text{score}_A$) | 25.3% |
| True regression rate | 4.7% (only 33% of 14% regressions are true compression failures) |
| SkillsBench | 87/87 tasks pass (zero regression) |
| Cross-model retention | 0.965 mean across 5 models / 4 families |
| Cross-framework (OpenCode) | 0.944 retention |
| Cost | ~\$14–18 for 600 skills (~20–40 LLM calls/skill) |
A less-is-more effect emerges: compressed skills outperform originals by 2.8% on average ($p = 0.002$, Cohen's $d = 0.107$), particularly for longer/more verbose skills (+11.8pp for official skills). SkillReducer significantly outperforms LLMLingua, direct LLM compression, truncation, and random removal at equivalent token budgets ($p \leq 0.003$).
§3 架构 / 方法图 #
Paper's Figure 3, verbatim (caption: "Overview of SkillReducer. Stage 1 (top) compresses routing descriptions via simulated-oracle-driven delta debugging followed by real-environment validation. Stage 2 (bottom) classifies body content, applies type-specific compression, deduplicates references, and validates through faithfulness and task-based quality gates with a feedback loop.").
The pipeline operates as a build-time pass on each skill. Stage 1 targets the routing description: semantic clauses are segmented and the ddmin algorithm iteratively removes clauses while maintaining routing correctness against an adversarial candidate pool. The simulated oracle provides fast (~2s) feedback for ddmin's $O(n \log n)$ iterations; real-environment validation catches over-compression (40.8% of skills needed selective restore). Stage 2 restructures the monolithic body into a tiered architecture: a taxonomy classifier assigns each paragraph-level item to one of five content types, then type-specific compression and cross-file deduplication produce a compact always-loaded core plus on-demand reference modules.
The agent loop for the optimized skill: on invocation, only the compressed core rules enter the context window. When the agent encounters a task requiring deeper reference (background, examples, templates), it uses read_file tool calls (max 6 per task) to load specific on-demand modules, guided by the "when" trigger clauses annotated on each module.
Planning style: SkillReducer is a preprocessing framework, not an agent itself. The optimized skills work within a ReAct-style agent loop where progressive disclosure assumes the agent can issue tool calls to load references on demand. The budget is max 6 read_file tool calls per task for reference loading.
Error recovery: If Gate 2 fails, the feedback loop promotes failed non-core items back to core_rule (monotonically growing core set, guaranteed to terminate in $|\mathcal{I}| - |I_{\text{core}}^{(0)}|$ steps). If restore/promotion fails entirely, the system falls back to the original skill.
§4 作者证明 #
记号表 #
| Symbol | Meaning |
|---|---|
| $s$ | A skill, with components $s.d$ (description), $s.b$ (body), $s.R$ (references) |
| $\lvert \cdot \rvert$ | Token count (cl100k_base tokenizer) |
| $\text{Cost}(s)$ | Total token cost: $\lvert s.d \rvert + \lvert s.b \rvert + \sum_{r \in s.R} \lvert r \rvert$ |
| $U = \{u_1, \ldots, u_n\}$ | Semantic clauses segmented from a description |
| $U^*$ | 1-minimal subset found by ddmin |
| $\mathcal{O}(d, Q, C)$ | Simulated routing oracle: returns 1 iff description $d$ routes correctly for all queries $Q$ against candidate pool $C$ |
| $s_{\text{adv}}$ | Adversarial "shadow" skill — topically similar, functionally distinct |
| $b^*$ | Compressed core body (always loaded) |
| $R^*_{\text{used}}$ | Subset of on-demand references actually loaded for a given task |
| $\rho$ | Fraction of items classified as core rules (empirically 0.383) |
| $\alpha$ | Core-internal compression factor ($\approx 0.63$) |
| $p_j$ | Probability that agent loads reference module $j$ |
| $\mathcal{C}_\tau(b)$ | Set of operational concepts of type $\tau$ in body $b$ |
| $\text{score}_A(t)$, $\text{score}_C(t)$ | Task score with original (A) and compressed (C) skill |
方程物理意义 #
Eq. 1 — Total skill cost:
$$\text{Cost}(s) = \lvert s.d \rvert + \lvert s.b \rvert + \sum_{r \in s.R} \lvert r \rvert$$
Physical meaning: full context window consumption when a skill is invoked — every token of the description, body, and all references injected at once in the monolithic architecture.
Eq. 2 — Optimized cost with progressive disclosure:
$$\text{Cost}'(s) = \lvert s'.d \rvert + \lvert b^ \rvert + \sum_{r \in R^_{\text{used}}} \lvert r \rvert$$
Physical meaning: under the tiered architecture, only the compressed description, compressed core body, and actually requested reference modules consume context tokens. The key reduction comes from $R^_{\text{used}} \subseteq R^$ — most tasks only need a fraction of references.
Eq. 3 — Faithfulness constraint:
$$\forall \tau: \; \mathcal{C}_\tau(s.b) \subseteq \mathcal{C}_\tau(b^) \cup \bigcup_{r \in R^} \mathcal{C}_\tau(r)$$
Physical meaning: no operational concept from the original body is silently dropped — every concept is preserved somewhere, either in the always-loaded core or in an on-demand reference module. Per-type checking ensures concept coverage is verified independently for each content type.
Eq. 4 — Retention metric:
$$\text{Retention}(t) = \begin{cases} 1.0 & \text{if } \text{score}_A(t) = 0 \\ \min\!\left(\frac{\text{score}_C(t)}{\text{score}_A(t)}, \; 1.0\right) & \text{otherwise} \end{cases}$$
Physical meaning: measures functional quality preservation per task. Capped at 1.0 to ignore improvement (conservative). Defaults to 1.0 when the original skill already fails (no blame for compression).
Eq. 5 — Expected invocation cost:
$$\mathbb{E}[\text{Cost}'(s)] = \lvert s'.d \rvert + \alpha \cdot \rho \cdot \lvert s.b \rvert + \sum_{j=1}^{k} p_j \cdot \lvert r_j^* \rvert$$
Physical meaning: the always-loaded portion ($\alpha \cdot \rho$ of body) plus the probabilistically-loaded references. With empirical values ($\rho = 0.383$, $\alpha \approx 0.63$, $p \approx 0.30$), this yields $\approx 0.426 \cdot \lvert s.b \rvert$, a 57.4% expected body cost reduction.
6 checks #
| # | Check | Verdict |
|---|---|---|
| 1 | Formal model covers the main claim? | Yes — Eqs. 1–2 formalize cost reduction; Eq. 5 gives expected savings. Propositions 1–2 in Appendix E prove convergence and expected cost bounds. |
| 2 | Variables defined before use? | Yes — all symbols introduced in §IV with physical interpretations. |
| 3 | Assumptions stated? | Partially — Proposition 1(iii) requires Gate 2 monotonicity in coverage, acknowledged as not always holding (28.9% violate). Uniform $p_j$ in Eq. 5 instantiation is a simplification. |
| 4 | Proof technique adequate? | Yes for convergence (monotone set growth on finite universe). The expected cost model follows from linearity of expectation. |
| 5 | Numerical sanity? | Yes — empirical 26.8%–43.2% body cost reduction falls within the theoretical range (30.9% conservative to 57.4% best case). |
| 6 | Key limitation of the formal model? | Gate 2's dual role as both optimization signal and evaluation criterion means the 86% pass rate reflects optimization-to-criterion, not independent evaluation. SkillsBench (87/87) partially mitigates but has ceiling effect (D passes 86/87). |
Agent-specific formal notes: This paper has formal analysis (Propositions 1–2), unusual for agent papers. However, there is no formal success-rate model for the compressed skills' agent performance — only for the feedback loop's termination. The less-is-more effect ($p = 0.002$) is purely empirical with no mechanistic explanation. A formal model bounding how context length affects agent task success (building on lost-in-the-middle and irrelevant-context findings) could strengthen the claim.
Failure mode classification: Three classes — (1) skill obsolescence (original description also fails, 10.7%), (2) evaluation noise (17% of regressions), (3) true compression regression (33% of regressions = 4.7% of all skills). The dominant true failure mode is "example-as-specification" — examples implicitly defining expected behavior, moved to reference modules where they are unavailable without a tool call.
§5 实验与数据 #
Skill Ecosystem Analysis (§III) #
Paper's Figure 1, verbatim (caption: "Token-length distributions of descriptions and skill bodies across three sources.").
The bimodal quality problem is visible: Wild skills cluster at very short descriptions (26.4% missing entirely, 44.1% under 20 tokens), while SkillHub descriptions average 47.78 tokens but contain non-routing filler. Body token counts span 3 orders of magnitude, with SkillHub skills (5,922 mean tokens) substantially larger than Community (1,400) or Wild (2,796).
Content Taxonomy Validation (§III-B) #
Paper's Figure 2, verbatim (caption: "UMAP projection of skill body items with GMM clustering (k=5). Five content clusters emerge, suggesting that body content naturally separates into the taxonomy categories (silhouette = 0.393).").
The clustering independently validates the five-category taxonomy: core rule (38.5%), background (40.7%), example (12.9%), template (7.6%), redundant (0.3%). The moderate silhouette score (0.393) indicates meaningful but not perfect separation — expected given that natural language categories have fuzzy boundaries.
| Category | Count | % |
|---|---|---|
| Core Rule | 5,817 | 38.5 |
| Background | 6,156 | 40.7 |
| Example | 1,948 | 12.9 |
| Template | 1,141 | 7.6 |
| Redundant | 45 | 0.3 |
Paper's Figure 5, verbatim (caption: "GMM model selection. k=5 is the first local peak (silhouette = 0.393), matching the five-category taxonomy.").
The first local peak at $k = 5$ justifies the five-category choice; higher $k$ does not consistently improve cluster separation.
Token Reduction (§V-B, RQ1) #
Paper's Figure 4, verbatim (caption: "Token reduction achieved by SkillReducer.").
Key results: 48% mean description compression (56.5% for existing descriptions, 44.6% for generated-then-compressed). Body compression: 39% mean, scaling with skill size — tiny (<500 tok) 58.8%, medium (1–3K) 79.1%, large (>3K) 89.1%, extra-large (>10K) 95.8%. On SkillsBench: 359K → 84K tokens (75% per-task reduction).
Functional Quality (§V-C, RQ2) #
| Subset | D (no skill) | A (original) | C (compressed) | Pass rate |
|---|---|---|---|---|
| All (600) | 0.684 | 0.722 | 0.742 | 86.0% |
| Official (87) | 0.696 | 0.743 | 0.831 | — |
| Community (464) | 0.691 | 0.713 | 0.724 | — |
| Wild (49) | 0.619 | 0.758 | 0.773 | — |
| Non-ceiling (481) | 0.606 | 0.653 | 0.678 | 82.5% |
The less-is-more effect is strongest for Official skills (+11.8pp) which are longer and more verbose. C outperforms A with $p = 0.002$ and Cohen's $d = 0.107$.
Baseline Comparison (§V-C, N=50, same token budget) #
| Method | Score | Retention | p vs. SkillReducer |
|---|---|---|---|
| A (Original) | 0.921 | — | — |
| C (SkillReducer) | 0.909 | 0.949 | — |
| LLMLingua | 0.767 | 0.820 | <0.001 |
| LLM direct | 0.866 | 0.918 | 0.003 |
| Truncation | 0.791 | 0.845 | <0.001 |
| Random removal | 0.694 | 0.750 | <0.001 |
LLMLingua's low retention (0.820) reflects a fundamental mismatch: perplexity-based pruning discards linguistically predictable but operationally critical tokens.
Ablation (§V-D, RQ3) #
| Condition | Components | Score | Retention | ΔA |
|---|---|---|---|---|
| A | Original | 0.939 | 1.000 | — |
| C | Full pipeline | 0.926 | 0.987 | −0.012 |
| C3 | Ref dedup only | 0.944 | 1.000 | +0.005 |
| C1 | Desc. only | 0.925 | 0.985 | −0.014 |
| C2 | Body compress (no classify) | 0.882 | 0.940 | −0.056 |
| C4 | Compress all (no classify) | 0.863 | 0.919 | −0.076 |
Taxonomy classification is the key component: without it (C4), retention drops by 6.8pp. Reference deduplication (C3) is a pure gain — it outperforms the original (0.944 vs. 0.939).
Cross-Model Generalization (§V-E, RQ4) #
| Model | $\text{score}_A$ | $\text{score}_C$ | Retention |
|---|---|---|---|
| GLM-5 | 0.926 | 0.962 | 0.986 |
| DeepSeek-V3 | 0.936 | 0.963 | 0.978 |
| Qwen3-max | 0.931 | 0.930 | 0.955 |
| GPT-OSS-120B | 0.951 | 0.958 | 0.982 |
| Qwen2.5-7B | 0.920 | 0.893 | 0.939 |
| Mean | 0.933 | 0.941 | 0.965 |
Compressions produced by DeepSeek-V3 transfer to Qwen2.5-7B (7B, retention 0.939) and GPT-OSS-120B (flat injection without progressive disclosure, retention 0.982). Cross-framework evaluation on OpenCode yields 0.944 retention with less-is-more effect (compressed 0.764 vs. originals 0.751).
Stage 1 Routing Validation #
| Metric | Value |
|---|---|
| Direct pass (no recovery) | 252 (42.0%) |
| Pass after selective restore | 245 (40.8%) |
| Fallback to original | 39 (6.5%) |
| Obsolescence (original also fails) | 64 (10.7%) |
| Total routing preserved | 536/536 (100%) |
| Description compression (passed) | 65% mean |
The high restore rate (40.8%) confirms the two-phase design: Phase 1's simulated oracle is fast but overly optimistic, and Phase 2 catches over a third of over-compressed descriptions.
§6 论证链 #
| Step | Claim | Evidence | Depends on |
|---|---|---|---|
| 1 | Skills suffer systemic token inefficiencies across descriptions, bodies, and references | Empirical study of 55,315 Wild + 100 SkillHub + 620 Community skills: F1 (26.4% missing descriptions), F2 (only 38.5% actionable content), F3 (1.67M tokens in 100 skills' references) | §III, Table I, Table X, Fig. 1 |
| 2 | Skill body content naturally separates into 5 functional categories | GMM clustering ($k = 5$, silhouette 0.393) on embedding space independently validates the LLM-assigned 5-category taxonomy | §III-B, Fig. 2, Fig. 5 |
| 3 | Delta debugging with adversarial oracle finds 1-minimal routing-sufficient descriptions | Stage 1 achieves 48% compression; 100% routing preserved (536/536 non-obsolete skills); two-phase design catches 40.8% over-compressions via selective restore | §IV-A, §V-B, §V-C Table V |
| 4 | Taxonomy-driven classification enables safe body compression that flat methods cannot achieve | Ablation: full pipeline retention 0.987 vs. no-classification C4 retention 0.919 (6.8pp gap); LLMLingua retention only 0.820 at same token budget | §V-D Table VI, §V-C Table IV |
| 5 | Compressed skills improve agent performance (less-is-more) | C (0.742) > A (0.722), $p = 0.002$, $d = 0.107$; effect scales with body length; ref dedup alone (C3 = 0.944) outperforms original (A = 0.939) | §V-C Table III, §V-D Table VI |
| 6 | Benefits transfer across models, model sizes, and agent frameworks | Mean retention 0.965 across 5 models / 4 families; 0.939 on 7B model; 0.944 on independent OpenCode framework | §V-E Table VIII |
Chain logic: Steps 1–2 establish the problem and its exploitable structure. Step 3 solves the routing layer. Step 4 is the critical link: structure-awareness (not raw compression power) is what enables quality-preserving body reduction. Step 5 provides the counter-intuitive payoff — compression is not merely acceptable but beneficial. Step 6 shows the benefit generalizes beyond the development setup.
§7 实现 cross-reference #
[实现未公开] — The paper states the tool will be "released as an open-source build-time tool" but provides no repository URL at the time of this analysis.
核心技术壁垒 — 展开 #
The adversarial simulated oracle is the engineering bottleneck for replication. It requires: (1) constructing a candidate pool with 4 TF-IDF-selected distractors plus 1 LLM-generated "shadow" skill that is topically similar but functionally distinct (e.g., given "JWT authentication", generating "OAuth 2.0 token refresh"), (2) randomizing candidate order per query to prevent positional bias, and (3) calibrating the fast simulated oracle against real-environment validation. The 40.8% selective restore rate confirms the oracle is systematically over-optimistic — the two-phase design is load-bearing, not a convenience.
The taxonomy classifier's conservative fallback (default to core_rule after 3 failed retries) is equally critical: it ensures the system never accidentally evicts operationally critical content, biasing toward under-compression rather than over-compression.
关键实现细节 #
- Semantic clause granularity for ddmin: Descriptions are segmented into "semantic clauses" (each captures one routing-relevant concept) rather than sentences or words. This provides the right granularity for delta debugging — sentence-level is too coarse (cannot remove sub-sentence filler), word-level makes $O(n \log n)$ impractical. The segmentation itself uses an LLM call, adding a dependency on consistent clause boundary detection.
- Separate models for compression and evaluation: DeepSeek-V3 compresses, Qwen3.5 evaluates (separate API key/session). This prevents the compression model from gaming the evaluator. Cohen's $\kappa = 0.939$ on 1,646 overlapping tasks validates LLM judge reliability against deterministic code-execution verifiers.
LLM backbone requirements #
SkillReducer's progressive disclosure design requires the target agent to support read_file tool calls (or equivalent). GPT-OSS-120B lacks tool-calling support — for such models, the compressed core is injected flat (still beneficial at retention 0.982, but forfeits on-demand reference loading). Minimum model capability: Qwen2.5-7B (7B) achieves 0.939 retention, confirming that compressions from a strong model transfer to substantially smaller models. Sensitivity to backbone is low: cross-compressor evaluation (Qwen3-max, Qwen2.5-7B as compressors) yields retention 0.897 and 0.874 respectively, indicating the pipeline's structure-aware design drives quality preservation, not the compression model's capability.