What are tokens? The unit of computation behind every AI model explained

Anyone looking at an AI API bill for the first time almost always trips over the same line item: a price per "token." Not per word, not per request, not per minute. Understanding that billing, and being able to manage your own usage sensibly, starts with understanding what a token actually is and why this particular unit became the base currency of the entire industry.

Language gets broken into building blocks

A language model doesn't process text the way a human reads it. Before a request ever reaches the model, it passes through a so-called tokenizer, which breaks the text into smaller units and converts each one into a number. These units, tokens, are usually not whole words but frequently occurring word fragments. The English word "unbelievable," for instance, gets split into several pieces by many tokenizers, while short, very common words often pass through as a single token.

As a rough rule of thumb: about 100 tokens correspond to roughly 75 English words. German text tends to be more token-intensive because of its long compound words — a word like "Rechtsschutzversicherung" has to be split into several pieces, while an English equivalent stays more compact.

The actual reason for this detour

Why not simply treat every word as its own unit? Because the number of possible words, names, technical terms, and new coinages across every language on earth is practically unlimited. A model could never learn and process every single word individually.

The solution is a limited, reusable set of building blocks. Methods such as Byte Pair Encoding break text down until a vocabulary of typically 100,000 to 250,000 tokens emerges, regardless of how many languages or how much text a model ultimately needs to process. This manageable, finite set can be represented as numbers and compared against each other at scale on specialized computing hardware. That is precisely what makes training modern language models practical in the first place.

Every provider has its own vocabulary

An often overlooked point: there is no universal tokenizer shared by all models. Every provider trains its own vocabulary, often even different vocabularies for different model generations from the same company. The same input text can break down into seven tokens for one model and eleven for another, simply because the two tokenizers were trained differently. Anyone who wants to see exactly how a given model breaks down a piece of text can try this directly using freely available tokenizer tools in the browser, with no programming knowledge required.

Why a model's thinking costs money too

Token consumption becomes especially visible with so-called reasoning models, meaning models with a visible internal thinking process before the actual answer. Such models often generate an entire paragraph of intermediate reasoning before the final answer: checking what might be meant, how to classify the request, which interpretation seems most plausible. All of that is text, and all of that text counts as consumed tokens, even if the user ultimately only sees a single short sentence as the answer.

Many providers now even let users set this thinking depth directly, usually through levels such as low, medium, or high. A higher level means more thorough, but also considerably more token-intensive reasoning before an answer is formulated at all.

Two sides of the same bill: input and output

What actually determines cost is the distinction between input tokens, everything sent to the model, and output tokens, everything sent back. Input includes far more than just the user's question: system instructions that define the model's behavior and that users typically cannot see or change, attached documents, and, in longer conversations, the entire prior conversation history, which gets resent with every new message.

Output tokens cost noticeably more than input tokens at nearly every provider, often around five times as much. Claude Fable 5 currently costs $10 per million input tokens and $50 per million output tokens. Claude Opus 5 comes in at $5 input and $25 output per million tokens, considerably cheaper, but with the same one-to-five ratio between the two prices.

The point where a model starts to forget

Every model has a context window: a fixed upper limit on how many tokens can be processed at once. Once that limit is reached, older information has to be compressed or discarded. Anyone working with coding assistants like Claude Code occasionally sees this as an automatic summary of the prior conversation once the context window fills up.

For a long time, the assumption held that a larger context window automatically leads to better answers. Anthropic pushes back on that in its own writing on context engineering, coining the term context rot for it: as the number of tokens in the context window grows, a model's ability to accurately recall information from it declines, well before the window's technical limit is even reached. Context is described there as a finite resource with diminishing returns, similar to a limited attention budget. A larger window, in other words, does not automatically make a model more reliable.

A huge price gap between models

There is currently a vast gap between the most expensive and the cheapest available models. Claude Fable 5 charges $10 input and $50 output per million tokens, making it one of the most expensive publicly available models around. At the other end of the scale sit models like DeepSeek, which charge a fraction of that price for the same volume of tokens. This gap exists because more expensive models are built for complex, long-running, high-stakes tasks, while cheaper models are often entirely sufficient for clearly defined, simpler ones.

A practical approach to your own usage

Anyone who wants to manage costs sensibly should sort tasks by difficulty and stakes, not by gut feeling. A high-stakes, complex task, such as a deep change to a business-critical codebase, justifies a more expensive model with a high thinking effort. A simple, clearly defined task with precise instructions, on the other hand, can often be handled just as well by a considerably cheaper model.

One thing worth keeping in mind: high token consumption is not a mark of quality. Burning through many tokens says nothing about whether anything useful actually came out of it. What matters is the ratio between tokens spent and the actual result, not the raw amount consumed.

Conclusion

Tokens are the shared unit of computation behind every language model, regardless of provider. Understanding that tokenizers break language down into a limited, model-specific vocabulary, that internal thinking itself costs tokens, and that a larger context window does not automatically lead to better answers gives you the basis to plan your own AI usage deliberately, instead of being surprised by the monthly bill.