Token Efficiency Is a Skill: Getting More Out of Your AI Coding Models

If you use AI coding agents every day, you eventually notice something uncomfortable: the biggest lever on quality and cost usually isn't the model you picked. It's what you feed it.

Tokens are the budget everything else draws from. Every instruction file, every stale note, every "just in case" paragraph you leave in an AGENTS.md or a skill definition gets read on every single run, whether or not it helps. And the evidence is mounting that a lot of it doesn't help.

What the research says about context files

A recent paper from ETH Zurich, Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents? (Gloaguen et al.), put this to a real test. The authors evaluated coding agents on SWE-bench tasks with LLM-generated context files, and on real-world issues from repositories where developers had committed their own context files.

The headline result: providing context files did not generally improve task success rates, while increasing inference cost by more than 20% on average.

Two details from the paper are worth sitting with:

1. Repository overviews are not helpful. The "here's a map of our codebase" section that model providers recommend, and that most of us dutifully write, produced no measurable performance benefit. Agents are already good at exploring a repo; narrating its structure to them mostly burns tokens.
2. Agents do follow instructions. The context files weren't ignored — the agents complied with what they were told. The problem is that compliance isn't the same as improvement. Extra instructions steer behavior whether or not that steering helps the task at hand.

The authors' conclusion is the right one: any attempt to improve agent performance through context should be rigorously evaluated before deployment, not assumed to help because it feels thorough.

Treat your agents files like a hot code path

The practical takeaway is that AGENTS.md, CLAUDE.md, and their cousins are not documentation. Documentation is read occasionally by people. These files are injected into every agent run — they're a hot path, and they deserve hot-path discipline.

A few rules I try to hold myself to:

• Only include what the agent can't discover. Build commands with non-obvious flags, the one test suite that must run before commit, a deploy step that will hurt you if skipped. Not the directory tree, not the tech stack the agent can see in package.json.
• Cut anything that restates defaults. "Write clean code," "follow existing conventions," "add tests where appropriate" — the model already behaves this way. These lines cost tokens on every run and change nothing.
• Prune on a schedule. Instructions rot. That workaround for a bug fixed three months ago is still being read, and still steering the agent, on every task. Stale instructions are worse than missing ones because the agent will follow them.
• Apply the same standard to skills. Skill files with sprawling preambles, redundant examples, and defensive over-explanation impose the same tax. A skill should read like a good runbook: trigger, steps, constraints, done.

The 20% cost figure from the paper is the average overhead. If you're running agents dozens of times a day across a team, cluttered context files are a standing tax on every invocation — paid in dollars, latency, and attention the model spends on your boilerplate instead of the actual problem.

Match effort mode to the task

The same economy applies to how much thinking you ask for. Most modern models expose some form of effort or reasoning control — extended thinking budgets, effort modes, or simply the choice between a frontier model and a fast one.

The failure mode runs in both directions:

• Max effort on everything turns a rename-this-variable task into a philosophy seminar. You wait longer, pay more, and sometimes get a model that overthinks its way into unnecessary refactors.
• Low effort on everything sends a fast model into a gnarly concurrency bug, where it produces a confident, plausible, wrong patch — and you spend the savings debugging it.

A rough heuristic that has served me well:

Task	Effort
Mechanical edits, renames, boilerplate, formatting	Low — fast mode, minimal reasoning
Typical feature work, straightforward bug fixes	Default — the model's standard setting
Architecture decisions, subtle bugs, security review	High — extended thinking, frontier model

The point isn't the specific mapping. The point is that effort should be a decision, made per task, not a global setting you never revisit.

Ask for less, get more

There's a common thread here. Context files, skills, and effort modes all tempt you toward the same mistake: more feels safer. More instructions, more background, more thinking — surely the model can only benefit?

The AGENTS.md study is a useful corrective because it measured that intuition and found it wrong. More context didn't make agents better; it made them more expensive. The discipline that makes a great engineer — knowing what to leave out — turns out to apply directly to how we operate these tools.

So audit your AGENTS.md. Delete the repo overview. Cut every instruction you can't tie to a real failure it prevents. Pick your effort mode like you'd pick a tool from the toolbox. And when you're tempted to add a paragraph "just in case," remember: the agent will read it every time, follow it faithfully, and bill you for the privilege.

Lean context isn't a constraint on the model. It's how you get the most out of it.