Skill-α reinforcement learning method boosts agent task success by 6.7 points
Researchers propose Skill-α, an RL approach that generates agent skills through sequential edits evaluated by a rollback reward, outperforming heuristic baselines by up to 6.7 percentage points on downstream benchmarks.
Skill-α is a reinforcement learning method that generates agent skills by treating skill construction as a sequence of individually evaluable edits. The approach, authored by Junhao Shen, Zhanqiu Zhang, Yiwen Guo, and Hong Cheng, addresses a fundamental problem: skills lack a natural supervision signal for relevance or correctness. Their value emerges only when they improve agent behavior on downstream tasks. The method introduces a "rollback reward" that evaluates each edit by comparing how an agent performs on an anchored query under the original skill versus the edited version.
Experiments show Skill-α outperforms heuristic and pipeline-based baselines in both document-to-skill and experience-to-skill settings. When paired with GPT-4o as the execution worker, Skill-α improved average downstream success rates by 3.3 percentage points on CL-Bench and 6.7 points on tau2-bench. Ablation studies confirm that both the rollback reward mechanism and progressive generation strategy drive the gains. The preprint was posted to arXiv on August 4, 2026.
