Amazon scraps internal AI leaderboard after employees game system through tokenmaxxing

Amazon deletes devs' tokenmaxxing leaderboard to minimize costs

An unofficial initiative to promote the use of AI internally has come unstuck. Enterprises everywhere have been urging employees to adopt AI, with internal leaderboards springing up to show who has used the most AI tokens. Such games can backfire, though, as Amazon recently discovered. Kirorank, an unofficial leaderboard tracking usage of Amazon's Kiro AI tool, ranked workers according to their AI activity, but senior managers at Amazon found that employees were creating AI agents to carry out unnecessary tasks in an attempt to boost their scores, a practise known as tokenmaxxing, according to a report in the Financial Times. The tracker has now been taken offline.

Amazon bins an internal AI leaderboard for its Kiro employees, because they were burning through too many costly tokens

When it comes to businesses jumping on the AI bandwagon, they're fallen into roughly one of three camps: Those that shun it entirely, those that use it with caution, and those that have wholeheartedly grabbed the reins and slapped the horses. In the case of Amazon, it's been very much in the lattermost category, though it's perhaps regretted being so enthusiastic about AI with its employees, now that the bills have come due. As reported by the Financial Times (FT), Amazon has nixed the use of an internal leaderboard, which kept track of how much staff were using its own Kiro agentic AI development platform. According to FT's sources, the leaderboard ended up being somewhat spammed by users creating pointless agents (which burned through lots of tokens to run), allowing them to rise up the rankings. In the world of AI, tokens are small chunks of data. When algorithms process text or images, they don't operate on full sentences, words, or pictures; instead, they're converted and broken down into small chunks (aka tokens), which get crunched by GPUs. Up until fairly recently, the top AI companies such as OpenAI and Anthropic employed a relatively simple flat subscription model, but with costs ballooning ever higher, they've increasingly turned to pay-by-token models instead. Somewhat obviously, that's landed Amazon with some painfully pricey bills to pay. Of course, Amazon has nobody to blame for this other than itself, because it apparently introduced a three-line whip for its employees to use AI as much as possible. Heavily paraphrasing, it essentially went along the lines of 'use AI for your job or lose your job to AI'. Some staff possibly went all in on 'tokenmaxxing' out of spite, but I suspect a good number of them did it out of fear of redundancy, or simply to show that they were a good employee. The FT says that Amazon has confirmed the leaderboard has been dropped, though the specific wording is a tad more subtle than mine: "The beta dashboard was not a formal or approved tool, and has since been deprecated." With the AI sucking up vast quantities of money, but creating little in the way of any viable returns so far, pay-per-token models are likely to become increasingly commonplace. And if they ever all go that way, I suspect a good many companies will have a change of heart over the use of AI in the workplace.

Amazon Scraps AI Leaderboard After Employees Begin 'Tokenmaxxing'

The leaderboard "was created by a group of employees who wanted to drive awareness for how AI can accelerate work", Amazon said, adding that the company was focused on "operational efficiency". When Amazon started ranking employees based on their AI usage, some workers reportedly began "tokenmaxxing", which is useful of AI for unnecessary tasks in an apparent attempt to climb the leaderboard. The behaviour comes amid to adopt the technology after Amazon introduced targets for more than 80 per cent of developers to use AI each week. "One of the internal dashboards, called KiroRank, was recently created by a group of employees who wanted to drive awareness for how AI can accelerate work and was never intended to promote the use of AI for usage's sake. The beta dashboard was not a formal or approved tool, and has since been deprecated. We're focused on and sharing best practices to celebrate innovation and operational efficiency gains across the company, and we're proud of the way our teams are embracing this technology," Amazon said in a statement.