TL;DR
Wix ran 250 controlled evaluations comparing AI agent performance with documentation versus curated skills, finding well-maintained docs often outperform purpose-built skills.
Key Points
- Doc optimization alone improved CLI task completion from 67% to 87%, cut token usage by 35%, and reduced wall-clock time by 9%
- Skills-only runs achieved 78% completion on CLI tasks vs 85% for optimized docs, using 10% more tokens and 14% more turns
- Small skill errors (misaligned scaffolding, missing exports, best-practice bloat) erased performance advantages entirely, burning 39-94% more tokens
- API tasks showed different tradeoffs: skills saved 29% tokens but ran 31% slower due to MCP tool fragmentation across multiple sequential calls
Why It Matters
As AI agents become primary consumers of developer documentation, this data challenges the industry hype around specialized 'skills' and demonstrates that well-structured, agent-optimized docs are the foundation—skills work best as a caching layer for specific tasks. Platform teams can now make evidence-based decisions about documentation strategy rather than following trends.
Source: www.wix.engineering