What TokenEconomizer changes
TokenEconomizer is an open-source plugin that uses experience from earlier sessions in the same repository to reduce coding-agent costs. It does not modify the model or the agent. Instead, it adjusts initial instructions, available tools and responses to repeated errors through interfaces the agent already provides.
Microsoft Research published an overview of the system on November 16, 2026. It reports average savings of 18.4% to 30.9% across 277 SWE-Bench bug-fixing tasks evaluated with five agent–model combinations. These results are reported by the team that developed the plugin, not independently validated.
The problem arises when an agent returns to the same project repeatedly. Each session may resend instructions, tool descriptions and parts of the conversation history to the model. If the agent repeatedly rediscovers how to run tests or loads tools it never uses, that repeated work can increase token consumption and cost.
Three ways to reuse experience
After each session, TokenEconomizer analyzes its log and updates the repository-specific state. In later sessions it can apply three mechanisms: micro-skills, tool pruning and intervention in failure loops. The system is designed to revisit its decisions as new data arrives.
Micro-skills are brief notes about a project, such as a recurring test command. TokenEconomizer proposes notes based on observed patterns and adds them to the initial context only after several sessions confirm the pattern. If later information contradicts a note, the system can remove it.
Tool pruning excludes tools from future sessions when repository history suggests they are unlikely to be used. Basic tools for running commands and editing files remain available, and excluded tools can be restored. This can reduce the text sent on each turn, although the result depends on the cost of that text and on how the provider charges for its cache.
Intervention in repeated failures uses a judge model to assess whether the agent is stuck or still making progress. If the judge identifies a blockage, the agent receives a short note suggesting a different approach. In the version described by Microsoft Research, if the blockage continues, TokenEconomizer alerts the developer: it does not stop the work on its own.
Mechanisms and their roles
| Mechanism | What it does | Relevant limitation |
|---|---|---|
| Micro-skills | Adds brief notes about repository patterns confirmed by earlier sessions. | Information can become outdated and must be reviewable or removable. |
| Tool pruning | Hides tools that history suggests are unlikely to be needed. | Basic tools remain available; excluded tools can be restored. |
| Failure intervention | Uses a judge model to assess whether the agent is stuck and, if appropriate, suggest another approach. | The intervention described does not automatically stop the work. |
Results reported by the evaluation
The main evaluation covered 277 SWE-Bench tasks across six repositories and five agent–model combinations. TokenEconomizer started with no prior history in each repository. Microsoft Research reports an average cost reduction of 18.4% to 30.9% compared with the default configuration; the cost includes calls made by the plugin itself.
Pass@1—the proportion of tasks solved on the first evaluated attempt—increased in four of the five combinations. Microsoft Research describes overall performance as broadly stable. That description does not guarantee unchanged quality in every configuration or establish that every individual task improved.
Savings do not necessarily appear from the first task. The authors say early tasks may cost more while TokenEconomizer learns, and that in their evaluation cumulative cost fell below the default in every repository by the sixth task. That pattern applies to the repositories and tasks tested; it is not a universal rule.
The publication also reports results in other settings: costs were 16.0% to 21.2% lower on Web-Bench, a sequential web-development evaluation. In security audits, where each task was run in a new repository, the reported reduction was 6.4% to 19.6%. The opportunities to reuse experience differ across these settings.
Results reported by Microsoft Research
| Evaluation | Reported result | Scope |
|---|---|---|
| SWE-Bench | Average savings of 18.4% to 30.9%. | 277 tasks, six repositories and five agent–model combinations. |
| Pass@1 on SWE-Bench | Increased in four of five combinations. | Measures the proportion of tasks solved on the first evaluated attempt. |
| Web-Bench | Savings of 16.0% to 21.2%. | Sequential web-development test. |
| Security audits | Savings of 6.4% to 19.6%. | Tasks in new repositories, with less reusable experience. |
How to interpret the evidence
Microsoft Research’s publication presents results from a project involving Microsoft Research, Microsoft Security and the University of California, Berkeley. The evaluation supports the claim that the authors observed savings in their tests; by itself, it does not show that the same effect will be reproduced elsewhere. The page links to the technical paper and code, but the information summarized here does not establish an independent replication.
Savings measure cost, not execution time. Nor do the reported results show that TokenEconomizer guarantees correct code, fewer errors or identical performance in every case. Pass@1 increased in four combinations, but that metric does not fully describe code quality, time spent or other aspects of an agent’s work.
Repository experience is central to the method. If every task takes place in a different project, the system has fewer opportunities to learn reusable patterns. With little history at the outset, learning may take time to offset its preparation costs. SWE-Bench results should not be transferred directly to interactive work, private repositories or teams using different configurations.
The results should be read as a specific evaluation of task cost and success, not as a general guarantee of savings. To assess an installation in your own repository, compare the cost of equivalent tasks with and without the plugin, repeat the baseline and keep the model, agent and execution conditions consistent.
What to measure in your own repository
A useful comparison starts by defining what counts as a solved task and recording the cost of each session. Keeping the agent, model and task the same between the baseline configuration and the one using TokenEconomizer makes differences easier to interpret. Repeating tasks also helps, because costs can vary between runs.
The learning period matters: a test that ends after the first session does not measure the same behavior as a series of tasks in the same repository. Recording cumulative cost helps distinguish initial spending from later effects, without assuming that savings observed in one evaluation will appear in another project.
Cost should be accompanied by a check of the results. Lower spending is not enough if tasks stop being solved or the outcome that matters to the team gets worse. Tracking both cost and task success makes it possible to identify trade-offs between efficiency and performance, rather than treating fewer tokens as synonymous with a better result.
A practical comparison
A local evaluation can be organized as a controlled comparison between two configurations.
- 01Choose representative tasks from the repository and define in advance what will count as a correct result.
- 02Run each task with the usual configuration and with TokenEconomizer, keeping the agent, model and other relevant conditions the same.
- 03Repeat the baseline configuration to observe how much cost varies without the plugin.
- 04Record cost per task and whether the result meets the agreed criterion; also calculate cumulative cost as the system learns.
- 05Interpret savings together with task outcomes and the learning period; do not extrapolate the percentage to other repositories without testing them.
Availability and practical limits
Microsoft Research describes TokenEconomizer as an open-source plugin that works alongside existing coding agents without modifying their underlying models. It does not require its own API key and keeps what it learns on the developer’s machine. Calls the plugin makes to a model may consume usage through the agent, which is why the authors include those calls in their reported cost calculation.
The publication presents the project as an option for exploring whether repository experience can reduce redundant work. The benefit depends on the tasks, accumulated history and configuration. The published figures do not guarantee the same savings for every agent or task, and the primary source does not establish an independent verification of the results.
In practice, the decision is not just about the average percentage. It also depends on whether tasks recur in the same repository, how long the system takes to accumulate experience, and whether the savings offset additional calls and configuration work. A local measurement can clarify these points without assuming that SWE-Bench results automatically transfer to day-to-day work.
Keep exploring
Sources consulted
Corrections and transparency
If you spot incorrect or outdated information, send us a correction with the page and source we should review.
Submit a correction