The way humans and artificial intelligences (AIs) process information differs significantly. While humans read using mentally stored vocabulary, machines interpret smaller units called tokens, which are essentially blocks with numerical labels.
Tokens constitute the language understood by large language models (LLMs), such as ChatGPT, Claude, and Gemini. The process of converting human language to machine language is called tokenization, and the reverse, transforming tokens into something understandable, is detokenization.
This concept is directly linked to platform subscription costs, as processing these blocks requires energy consumption and computational capacity. Companies use the count of consumed tokens—both in the request and the AI's response—to determine system costs, set memory limits for each model, and charge for the service.
For an AI to understand a message, whether it is text, audio, or image, it needs to segment it into blocks, or tokens. According to Fabrício Carraro, Program Manager at Alura, a token can be understood as a subword. He exemplifies that words like 'feliz' (happy), 'triste' (sad), and 'divisível' (divisible) can be individual tokens.
The reason they are called subwords relates to compound words; for example, in 'infeliz' (unhappy) or 'indivisível' (indivisible), the token 'in' is used. This token 'in' has a specific number in the model's vocabulary, just like the token 'feliz' (number 352). Together, the token 'in' and the token 'feliz' form the word 'infeliz'.
At the core of the AI, the LLM, it receives a sequence of numbered blocks (the prompt) and executes complex calculations to predict which output blocks should be (the AI's response). However, when interacting with platforms like ChatGPT, the user does not access the engine directly, but rather an external software layer called a harness.
The harness functions as a steering system for the LLM, coordinating the flow of tokenization and detokenization. Carraro explains that this component takes the user's question and breaks it down into possible secondary questions to formulate the answer. Instead of giving an immediate answer, it internally generates and resolves these sub-questions, and only after concluding this internal reasoning is the final answer presented to the user, which is when token usage occurs.
Another relevant factor in AI interactions is the context window, which is intrinsically linked to tokens. This window can be compared to the space on a workbench, representing the AI's 'short-term memory,' and the tokens would be the papers with information spread across that table.
More technically, the context window establishes the maximum number of tokens (summing input prompts and AI output responses) that the model can process simultaneously. If the conversation remains within this limit, the AI retains the memory of the entire dialogue. If the limit is exceeded, some information is lost.
Although the token is associated with AI subscription costs, such as ChatGPT's Go plans and Gemini's Plus, it is not the currency of the AI itself. It is the unit of data processing measurement, serving as a consumption metric. Companies assess computational effort and operational costs based on the volume of tokens entering (prompts) and leaving (responses).
For example, with Claude Fable 5, Anthropic charges $10 per million input tokens and $50 per million output tokens. Carraro emphasizes that although one million tokens seems large, for a programming problem, this can be consumed in just a few prompts, depending on complexity. He adds that for most codebases used, this volume is not excessive, but output costs can increase rapidly.