How Anthropic Writes Skills

Last week Anthropic published lessons learnt of how they build agent skills internally. It's quite interesting to read recommendations from the company that introduced the concept in the first place.

Key ideas:
🔸 Don't be obvious. Model already knows how to code. A skill should provide instructions that change default agent behavior, not repeat the data the model was trained on.
🔸 Build a gotchas section. Add common mistakes and lessons learned. This helps the agent avoid repeating the same failures.
🔸 Use progressive disclosure. A skill is not just a SKILL.md. It can include additional files that are loaded on demand, reducing context overload.
🔸 Don't be too specific. Give the agent information it needs, but leave the flexibility to adapt to the situation.
🔸 Separate configuration from instructions. Store setup data in config.json or collect required input from the user.
🔸 Write description for the model, not for humans. A description should help the model to understand when the skill should be invoked.
🔸 Use long-term memory. Skills can maintain their own data in a subdirectory and reuse it across executions.
🔸 Automate where possible. Not everything should be a prompt. Some actions can be automated with helper scripts and functions.

Unfortunately, the article doesn't provide any guidelines of how to evaluate skill effectiveness. It's still not clear how to understand if a skill actually works, how to compare two versions of the same skill, or how to detect that a skill is no longer useful.

Provided recommendations are based mostly on observations of how popular internal skills are structured. It's useful but not measurable.

So despite the fact that skills are the most powerful agent extension now, evaluating them remains on of the hardest engineering problem.

But that's another story.

#ai #engineering