🔍 Read the full analysis: Are Token Counts Useful AI Metrics? Amazon’s Bets Add Some Context on ThorstenMeyerAI.com
Get hardware and tech essentials delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
Amazon published updated engineering guidance on October 27 warning that token counts and other AI-use metrics can distort behavior when used as targets. The memo says those measures can still help identify adoption patterns, but engineers should be judged on customer, business and team outcomes. Amazon’s Nova and Anthropic investments provide a wider, though not directly comparable, backdrop.
Amazon published updated guidance for software engineers on October 27, warning that token counts and other measures of AI use can create perverse incentives when treated as performance goals. The memo says teams should focus on customer, business and team outcomes, a distinction that comes as the company’s own AI models and its investment in Anthropic have followed different paths.
The guidance was issued on Amazon’s news site by Terk Terkowitz, a vice president and distinguished engineer in the company’s stores business. Written by senior engineers, it updates role guidelines for software engineers and avoids defining the job around specific AI tools. The memo says current descriptions of AI work could quickly become outdated, so it anchors the role in impact rather than tool use.
Amazon calls measures such as token usage, AI-tool adoption, code changes and reviews “deeply flawed” when used as targets. The memo says turning them into goals can raise costs and make systems more complex. It gives two examples: “tokenmaxxing is at odds with frugality” and measuring lines of code is at odds with “invent and simplify.”
The guidance does not say those metrics are useless. It says they currently “correlate well” with outcomes that matter, and that teams adopting AI most are delivering more while teams that do not adopt it are falling behind. Amazon’s stated distinction is between using activity data to understand patterns and scoring people on how much AI they use.
Six months at Amazon
From token leaderboards to an outcomes memo
Amazon spent 2026 learning the same lesson twice: once with how its engineers use AI, once with its own AI models.
- May 2026Token leaderboards backfireFinancial Times (via The Decoder): staff give an internal agent tool pointless tasks to inflate token counts. Reported, anonymous sources.
- Late May 2026KiroRank “deprecated”Amazon spokesperson to Business Insider. SVP Dave Treadwell: “Please don’t use AI just for the sake of using AI.”
- 24 Jul 2026San Francisco AGI Lab site closesConfirmed by Amazon to GeekWire. Frontier model research continues under Pieter Abbeel.
- 28 Jul 2026Most flagship Nova models wound downBusiness Insider, people familiar: Premier, Omni, Reel, Canvas to “keep the lights on”. Amazon: models “remain one of the most important things we’re working on”.
- 14 / 30 Sep 2026Nova Sonic v1, Nova Reel, Nova Canvas reach end of lifeAWS Bedrock model lifecycle page. Requests to them now fail.
- 26 Oct 2026Anthropic files its S-1Amazon notes: $6.2 billion principal, 12% ownership cap that Amazon can waive. Anthropic commits at least $110 billion to Amazon through April 2036.
- 27 Oct 2026The memo: outcomes, not tokenmaxxingUsage and throughput metrics are “deeply flawed” as targets. “Tokenmaxxing is at odds with frugality.”
- 29 Oct 2026Amazon’s third-quarter resultsAnnounced by Amazon. Not yet published when this article went live.
Sources: Amazon engineering memo (27 Oct 2026); Business Insider (29 May and 28 Jul 2026); GeekWire (24 Jul 2026); The Decoder (12 May 2026, summarising the Financial Times); AWS Bedrock model lifecycle (read 28 Oct 2026); Anthropic S-1 (26 Oct 2026).
Powered by Thorsten Meyer AI
Why Amazon Rejects Usage Targets
The memo addresses a practical problem for employers adopting generative AI: activity is easier to count than impact. Tokens, code changes and tool logins can be gathered quickly, while whether a product improved for customers or a team delivered better work may take longer to establish. Amazon’s guidance warns that substituting the easy count for the desired result can reward wasted work and add cost.
It also offers a management approach rather than a new numerical target. The memo tells leaders to examine both low and high adoption: investigate why some teams are not finding AI useful, and check whether heavy users are actually delivering more. It says engineers who find that current AI tools do not help their work should record and share that finding, not be treated as failing to adopt.
The company’s wider AI record is relevant but does not prove that the memo was prompted by the performance of any particular product or investment. Reporting on Nova and the terms of Amazon’s Anthropic stake show that AI activity and business returns are different questions. The memo’s direct focus is how Amazon evaluates engineering work.
Products Worth Considering
LG 24G414B-B 24-Inch UltraGear Full HD IPS Gaming Monitor
LG UltraGear 45GX900A-B 45-Inch Curved OLED Gaming Monitor
LG 34GX90SA-W 34-inch Ultragear WQHD OLED Curved Gaming Monitor
As an affiliate, we earn on qualifying purchases.
From KiroRank to Formal Guidance
The memo follows earlier reports of informal usage tracking inside Amazon. In May, the Financial Times reported that employees were assigning unnecessary work to an internal agent tool, MeshClaw, to increase token totals. The report also described a weekly AI-use target and internal leaderboards. Those accounts were attributed to employees and are reported claims, not a company-confirmed account of all staff behavior.
Amazon did confirm to Business Insider at the end of May that a leaderboard called KiroRank had been deprecated. The company said it was an informal tracker built by employees and “was never intended to promote the use of AI for usage’s sake.” Senior vice president Dave Treadwell also told staff, “Please don’t use AI just for the sake of using AI.” The October memo moves the point into formal engineering role guidance.
The source material also describes contrasting corporate AI bets. Business Insider reported in July that most flagship Nova models were moved to “keep the lights on”; Amazon’s lifecycle information lists Nova Canvas, Nova Reel and the first Nova Sonic as retired. Separately, Amazon’s June 30 filing reportedly valued its Anthropic holdings at about $190 billion, against roughly $18 billion invested. Those figures are reported in the source material and should not be read as a direct measure of product performance or cash returns.
“Tokenmaxxing is at odds with frugality.”
— Amazon engineering memo, published October 27
What the Metrics Cannot Show
The memo does not set out a single replacement formula for measuring AI’s effect on engineering output. It acknowledges that long-term outcomes are slow and hard to measure, while saying input metrics correlate with results at present. It does not provide data supporting the strength of that correlation, define “delivering more,” or quantify how much AI use changes results.
It is also unclear how consistently managers will apply the guidance across Amazon teams, or whether the company will publish follow-up measures. The May reports about unnecessary tasks and usage targets rely partly on anonymous employee accounts; Amazon’s confirmation that KiroRank was discontinued does not independently verify every reported practice.
The source material’s valuation figures for Anthropic are paper valuations, not evidence of realized cash proceeds or a like-for-like comparison with Nova. Amazon’s upcoming quarterly results may provide financial context, but the company has not, in the material provided, connected those figures to the engineering memo.
Amazon’s Results and Adoption Review
Amazon is scheduled to report third-quarter results on October 29, one day after the memo’s publication. Those results may add information about the company’s overall business and AI-related spending, though they will not necessarily answer whether AI tools improve engineering outcomes.
Under the memo, managers are expected to examine adoption patterns at both ends of the range, use AI tools in their own production work, and explain where the tools helped or did not. The next test is whether teams use that process to improve decisions without turning usage counts back into quotas. No separate timetable for a company-wide evaluation or further changes to the guidance was provided in the source material.
Key Questions
What did Amazon announce?
Amazon published engineering guidance on October 27 saying that token counts and other AI-use measures should not be performance targets. It says engineers should be evaluated on their impact on customers, the business and their teams.
Does Amazon say token data should not be tracked?
No. The memo says usage measures can help show adoption patterns and currently correlate with important outputs. Its warning is against using them as targets instead of measures.
What was KiroRank?
KiroRank was an informal employee-built leaderboard tracking AI use. Amazon told Business Insider it had been deprecated and was not intended to encourage AI use for its own sake.
What happens if an engineer finds AI tools do not help?
The memo says that is “a finding, not a failing.” It asks engineers to write down why the tools did not help and share the information with colleagues and managers.
When are Amazon’s next quarterly results due?
Amazon is scheduled to report its third-quarter results on October 29, 2026. The results may provide business context, but the source material does not say they will include a direct measure of AI’s effect on engineering work.
Source: ThorstenMeyerAI.com
