As artificial intelligence applications move beyond simply answering queries to performing agent functions, writing code, and controlling physical systems, token economics is becoming a growing concern for businesses and developers.
Jigar Khalani, Senior Director of Enterprise Solutions Architecture and Engineering at NVIDIA South Asia, detailed the emerging 'token economy' at the YourStory's DevSparks Hyderabad 2026 event, dividing it into four components: utility, demand, supply, and monetization.
He emphasized that the key principle is that the problem arises when resources are overspent, which prompts thoughts about saving.
Token consumption shows rapid growth. Khalani reported that NVIDIA initially planned to use 16 trillion tokens in 2026, but by August, this figure had already exceeded 18 trillion, not including token usage through tools like Microsoft Copilot.
For developers, according to Khalani, the primary question is not choosing the largest model, but determining the most suitable model for a specific task. He noted that a simple query does not always require a large model.
When moving to more complex tasks, significant latency issues arise, required throughput decreases, and the type of necessary model becomes much more complicated.
Thus, selecting models based on workload type rather than using one universal model can become an important part of managing AI costs.
The demand side becomes more complex as organizations transition to agentic AI. An AI application can generate tokens not only in response to the initial user query but also during reasoning, feedback, tool calls, and the agent's iterative cycles.
Khalani advised IT leaders (CIOs and CTOs) to view token demand through a relatively simple formula: number of concurrent users, requests per user, and average tokens consumed per request. Furthermore, organizations must consider workload types, usage patterns, and the possibility of serving requests via cache instead of generating new tokens.
He added that the more cache that can be used instead of creating a new token, the more costs can be reduced, at least by five or eight parts. This, he said, is the current market standard for assessing potential savings in token economics.
Managing supply, Khalani explained, requires optimization at three levels: model efficiency, system efficiency, and software efficiency. He stated that the days when system and hardware knowledge was acceptable for a resume are over, asserting that developers increasingly need to understand the underlying systems of their applications. 'A new era has dawned.'
This implies the necessity of understanding memory, network connections, data storage, and workload distribution across GPUs. For example, with large models, the question is not just the number of GPUs used, but how they interact with each other, and whether they should be placed in the same or different racks.
This shift could potentially affect the laptop itself. Khalani suggested that companies might begin evaluating the feasibility of running small AI models locally on employee devices, which would reduce the need to send every request to an API, cloud, or internal AI factory.
However, for Khalani, a more significant opportunity lies beyond simply reducing token costs. He believes the future is not about how many tokens are used or how much efficiency increases, but what data is fed, what model is trained or fine-tuned, making it smarter to provide a more intelligent token in response.
This could lead to what Khalani termed 'customer stickiness.' The better the AI system understands a returning customer, the more valuable it becomes to them, and perhaps the more willing they are to pay for it.
'This is where token monetization will come into play, and it will be the most profitable business you can run,' he concluded.
As agentic and physical AI expands, the underlying infrastructure will become increasingly complex. For developers, the boundaries between software, models, hardware, and economics are beginning to blur.