The Chinese company MemTensor has introduced and made publicly available a series of models it calls memory-oriented foundation models, named MemT 1.0. This series includes two models: MemT-1.0-Flash, which has a total of 288 billion parameters, of which 17 billion are actively used, and MemT-1.0-Nano, which has 36 billion total parameters and 3 billion active ones.
The weights of these models are available on the Hugging Face platform, and the corresponding code is hosted on GitHub along with a technical report.
The models are designed to solve a common problem associated with long dialogues. Most existing systems store the entire history in the context window, which leads to slowdowns and increased inference costs with every new turn. Instead, MemT integrates a trainable memory module directly into the model.
After each interaction, the model records new information into a fixed memory state and retrieves it when necessary, instead of re-reading the entire previous log every time.
MemTensor reports that the training of the models was conducted in two stages based on 34 billion tokens: first, the model learned to write information into memory, and then it learned to answer questions using this stored information.
Performance Results and Comparison
All presented performance metrics are based on the company's own evaluation. MemTensor claims that end-to-end inference runs approximately 1.86 times faster than the full context approach with a history of 128 thousand tokens, and about 3.99 times faster with 256 thousand tokens.
The time to the first token response is estimated at approximately 104–112 milliseconds for the memory model, compared to approximately 445 milliseconds for full context at 256K. Furthermore, the company notes that the memory state size per session with a 256K history is about 40 MB, which is significantly less than the 5.06 GB required for the full context cache, representing a reduction of approximately 130 times.
On 11 memory benchmarks, MemTensor reports average scores of 50.6 for Flash and 47.0 for Nano. These results place the models first and second among comparable parametric memory methods and roughly match the results of retrieval-augmented approaches on average, even without retrieving history during the query.
The company also emphasizes that adding memory did not degrade the overall capabilities of the Flash model under tests with a high number of distractors.
Developers should note the licensing terms: the repository code is published under the non-commercial PolyForm license, while the model weights have Apache and MIT licenses, respectively.
The team plans further development towards improving long-term memory, increasing memory capacity, and reducing memory update costs, with the long-term goal of creating agents that continue to learn based on their interactions.