WeChat has begun conducting a soft launch of Xiaowei—a native artificial intelligence assistant integrated directly into the application. This assistant differs from standalone AI applications like Yuanbao. Users can access Xiaowei via an entry point in the upper left corner of the WeChat homepage using text or voice commands. Subsequently, Xiaowei utilizes WeChat's functions and services to complete the assigned tasks. It functions more as a consumer agent embedded within the WeChat ecosystem rather than a typical chatbot.
The core of Xiaowei's operation is the WeLM model, which the WeChat AI team has been developing for many years. This model is not new; in 2022, the WeChat AI team introduced WeLM—a Chinese pre-trained language model with 10 billion parameters. At that time, it demonstrated high performance across 18 Chinese tasks, matching or surpassing larger models in certain tests.
The version of WeLM used in Xiaowei today has undergone several updates. The team converted WeLM to a sparse mixture-of-experts architecture. The WeLM-80B model has a total of 80 billion parameters but activates only about 3 billion per token. It was trained on less than 14 trillion tokens and possesses reasoning capabilities, multilingual understanding, and a 128K context window. This is critically important for a product with WeChat's scale, which processes enormous volumes of requests daily, because WeChat needs an AI infrastructure accessed frequently by hundreds of millions, if not billions, of users, rather than a large model used occasionally.
The team did not stop at 80 billion parameters. A WeLM-617B version has emerged, boasting a total of 617 billion parameters and activating 23 billion per token, also built on the MoE principle. Instead of simply increasing the number of parameters, the WeChat team is exploring a different scaling method called Hidden Decoding. Traditional scaling either increases the model size or allows it to think longer before responding. Hidden Decoding keeps the structure of the main part of the model largely unchanged but performs more internal computations per token by expanding one token into multiple streams along the sequence dimension. This allows the model to utilize additional hidden computations to improve reasoning without adding Transformer layers or increasing width.
To control computational costs, the team developed the Stream-Factorized Attention mechanism, which reduces attention overhead from quadratic to nearly linear growth as the number of streams increases. These mechanisms are ideal for WeChat's massive user base. Thus, Xiaowei, combined with WeLM, suggests that WeChat may not need to create another super AI application; it might integrate AI directly into the operational layer of WeChat itself.
