SenseTime releases SenseNova-Loop-9B-Preview model with cyclic architecture

Rock Matrix, in collaboration with the University of Science and Technology Hong Kong and Carnegie Mellon University, has released the EdgeRock-4B model to the public. This small language model is characterized by its constant memory consumption regardless of context length.

The base model for EdgeRock-4B is Alibaba's Qwen3.5-4B, according to information presented on the project page on GitHub and in the corresponding arXiv article, as reported by QbitAI.

The EdgeRock model was not trained from scratch; the team performed a phased conversion of Qwen3.5-4B. 24 layers of Gated DeltaNet, which already contain a fixed-size state, were preserved. Furthermore, 8 full attention layers, which typically store keys and values for every previous token, were converted into a sliding window attention mechanism covering the last 1024 tokens.

To prevent forgetting information outside this window, developers implemented 'Rock Memory Anchors.' For each KV head group, the model saves the 1024 most useful out of 256 candidate tokens in a small anchor bank.

As a result, it was possible to achieve a fixed KV cache size of 72 MB at a 256K context, representing a massive reduction compared to the 8 GB required by Qwen3.5-4B, which is a 99.1% decrease. The full state for the sequence is 97.76 MB.

When running on a single 80 GB Hopper GPU, EdgeRock provided service for 512 concurrent streams at a rate of 12,124 tokens per second, which is 6.42 times higher than Qwen3.5-4B (1,889 tokens per second) under a batch limit of 64. The maximum memory consumption was 61.26 GB. When processing 128K input tokens and 128K output tokens, EdgeRock maintained a stable speed of 233.7 tokens per second, whereas Qwen slowed down from approximately 169 to 151.3 tokens per second. GPU runtime decreased to 24.9% of Qwen's time for a 4K input and to 15.6% for a 32K input.

According to the team, the accuracy trade-off is minimal. EdgeRock retains 96.3% of Qwen3.5-4B's AIME score and 97.5% on MATH-500, surpasses it on 8 benchmarks, and maintains 95.3% vision metrics. Its average pass@1 score on AIME from 2024 to 2026 is 82.9%, compared to 18.8% for the strongest tested limited-state model, RWKV-7 G1j 13.3B. Additionally, it ranks first among limited-state models across 24 out of 29 text benchmarks. In an ablation study, the information retrieval task showed 94.1% accuracy using memory anchors versus only 0.4% without them.

The model weights accept both text and image input and are available in quantized builds. They are released under the EdgeRock model license, which, according to the team, permits commercial use. Larger versions are planned for release.

The Chinese company MemTensor has introduced and made publicly available a series of models it calls memory-oriented foundation models, named MemT 1.0. This series includes two models: MemT-1.0-Flash, which has a total of 288 billion parameters, of which 17 billion are actively used, and MemT-1.0-Nano, which has 36 billion total parameters and 3 billion active ones.

The weights of these models are available on the Hugging Face platform, and the corresponding code is hosted on GitHub along with a technical report.

The models are designed to solve a common problem associated with long dialogues. Most existing systems store the entire history in the context window, which leads to slowdowns and increased inference costs with every new turn. Instead, MemT integrates a trainable memory module directly into the model.

After each interaction, the model records new information into a fixed memory state and retrieves it when necessary, instead of re-reading the entire previous log every time.

MemTensor reports that the training of the models was conducted in two stages based on 34 billion tokens: first, the model learned to write information into memory, and then it learned to answer questions using this stored information.

Performance Results and Comparison

All presented performance metrics are based on the company's own evaluation. MemTensor claims that end-to-end inference runs approximately 1.86 times faster than the full context approach with a history of 128 thousand tokens, and about 3.99 times faster with 256 thousand tokens.

The time to the first token response is estimated at approximately 104–112 milliseconds for the memory model, compared to approximately 445 milliseconds for full context at 256K. Furthermore, the company notes that the memory state size per session with a 256K history is about 40 MB, which is significantly less than the 5.06 GB required for the full context cache, representing a reduction of approximately 130 times.

On 11 memory benchmarks, MemTensor reports average scores of 50.6 for Flash and 47.0 for Nano. These results place the models first and second among comparable parametric memory methods and roughly match the results of retrieval-augmented approaches on average, even without retrieving history during the query.

The company also emphasizes that adding memory did not degrade the overall capabilities of the Flash model under tests with a high number of distractors.

Developers should note the licensing terms: the repository code is published under the non-commercial PolyForm license, while the model weights have Apache and MIT licenses, respectively.

The team plans further development towards improving long-term memory, increasing memory capacity, and reducing memory update costs, with the long-term goal of creating agents that continue to learn based on their interactions.

At the 10th CSDI summit in Shenzhen on October 16, SenseCore Vice President and CTO of SenseTime, Xuan Shanming, discussed how the company is restructuring its inference infrastructure for the age of agents. In this new era, the demand for tokens significantly exceeds the capabilities of chatbots due to multi-agent workflows and long tasks performed without constant supervision.

The main focus of the presentation was not on the number of accelerators in the data center, but on the ability of this center to stably produce high-quality tokens within specified cost and service level constraints.

The SenseCore solution is based on separating the Prefill and Decode (PD) processes within heterogeneous clusters of 10,000 GPUs located in their AIDC in Lingang. The Prefill process handles input tokens and is computationally intensive, while Decode generates output tokens by constantly accessing the context cache and requiring high memory bandwidth.

In the SenseCore architecture, domestic accelerators are focused on executing Prefill, while high-performance GPUs handle Decode. These stages exchange state via KV Cache channels, which cover mixed factories with bandwidths of 800G, 400G, and 200G within a single cluster.

This separation creates an operational risk: in some configurations, three to fifty low or medium-level Prefill nodes may serve one Decode node, meaning that a failure of a Decode node could halt a large group of Prefill nodes. To minimize this risk, SenseCore maintains a small reserve of high-end cards in the Prefill pool for quick switching, and also allows nodes to dynamically change roles between Prefill and Decode depending on the load.

During one test scenario using the same number of flagship high-performance GPUs as the baseline configuration, the company reported approximately a fivefold increase in overall token throughput after implementing heterogeneous PD scheduling. Furthermore, the company is exploring cross-domain PD, directing Prefill work to regional medium-level capacities and Decode work to remote high-bandwidth clusters.

This becomes relevant against the backdrop of the decreasing size of the KV Cache for million-token contexts—it shrinks from nearly 100GB to about 1GB on some models, as well as due to the separation of Attention/FFN within the Decode process.

SenseCore claims that its platform currently processes about 6 trillion tokens daily and aims to reach 10 trillion tokens per day by the fourth quarter of 2026, with a projected annual growth in total tokens of approximately 25 times. It should be noted that these throughput and speed increases are company statements. During the presentation, it was emphasized that domestic chips are intended to take up a larger share in Prefill and mixed inference tasks, rather than completely replacing the high-performance GPUs used for Decode.