The Xiaohongshu Engine team presented the HELMSMAN system at the OSDI 2026 conference. This system is a cost-effective, high-performance solution for vector approximate nearest neighbor search, specifically designed for NVMe SSD arrays on fully flash memory.
HELMSMAN replaced Xiaohongshu's previous infrastructure, which consisted of approximately 35,000 CPU cores and 350 TB of DRAM across only about 40 fully flash servers. This achieved a reduction in hardware costs by over 90% while maintaining millisecond latency for company services such as search, recommendations, and advertising. These services process hundreds of billions of high-dimensional vectors with millions of queries per second.
The core idea is that clustered ANNS methods are a more suitable architecture for modern high-speed SSD arrays compared to graph-based ANNS methods. Graph systems utilizing DRAM and SSD, such as DiskANN, suffer from sequential I/O dependency because reading from one SSD determines the next visited neighbor, which hinders SSD bandwidth saturation. Clustered systems, conversely, read candidate clusters in parallel, aligning with the parallel I/O capabilities of SSDs.
Traditional SPANN clustering systems faced three issues in production environments: insufficient throughput due to Linux I/O stack overhead, non-adaptive search strategies that either scanned too much or too little depending on query complexity, and slow single-processor index building that could not scale to hundreds of billions of vectors. HELMSMAN overcomes these obstacles through three innovations:
First, it features a specialized storage stack for ANNS that bypasses standard Linux I/O paths by managing NVMe queues directly in user space. This reduced software overhead, which previously consumed up to 58% of total latency. Second, a hierarchical learning mechanism for search pruning was implemented, which adaptively predicts the optimal number of clusters to scan based on query complexity, top-k size, and centroid distribution. Third, a distributed pipeline for index building with GPU acceleration was introduced, allowing indexes to be rebuilt at the scale of billions of vectors in just a few hours, matching the minute-level model updates required for recommendation and advertising systems.
Production tests demonstrated the system's efficiency. HELMSMAN achieves a 2–16 times improvement in throughput compared to existing DRAM-SSD systems, including DiskANN and SPANN, reaching up to 85% of pure in-memory deployment throughput while meeting SLA targets for average and tail latency within milliseconds. The ability to rebuild indexes at the scale of 10 billion vectors in hours makes HELMSMAN practical for production environments where embedding models and vector data are frequently updated. Xiaohongshu's deployment spans numerous business areas, including search, recommendations, advertising, content safety, and RAG pipelines.
The implications for cost and scalability extend beyond Xiaohongshu. Vector search infrastructure is one of the fastest-growing cost centers for internet platforms, as embedding retrieval is becoming universal for search, recommendations, and AI-powered features. Dependence on DRAM was a structural factor driving cost growth, given that vector scales double annually. HELMSMAN proves that high-performance ANNS on standard flash storage is viable in production even for the most latency-sensitive online serving scenarios, potentially changing the infrastructure economics for the entire industry. The research also raises questions about whether other memory-bound AI components can benefit similarly from co-designing the storage stack instead of simply expanding DRAM.
XPower Technology has unveiled the RPP AE7100E architecture chips at the WAIC event. These nail-sized microchips are capable of supporting models with parameters ranging from 400 billion to 1.6 trillion by utilizing distributed computing, while maintaining minimal token cost.
XPower Technology has established itself as a significant player in the domestic AI computing market at WAIC 2026. The company demonstrated its matrix of artificial intelligence accelerators based on its proprietary Reconfigurable Parallel Processor (RPP) architecture. This architecture covers cloud, edge, and end computing scenarios. The company's core idea directly challenges the industry's prevailing doctrine regarding the necessity of using giant chips.
The AE7100E chips, which are about the size of a fingernail, can be integrated in quantities of 12 per accelerator card. This configuration allows for running large language models with 400 billion to 1.6 trillion parameters through distributed computing, achieving an industry-optimal cost efficiency per token. XPower CEO, Li Yuan, noted that the shift from training-oriented computing to inference-oriented computing is a key trend. He emphasized that AI programming has become a critical use case for inference, as the company's experience has shown that AI can now perform CPU compiler development, which previously required a professional team for years, and this capability can be scaled for rapid application across various tasks.
Since 55% of model requests in global data centers are directed towards open-source models, the inference market is accessible not only to a few giants but to any enterprise, creating huge demand for cost-effective inference infrastructure. A key observation is the inefficiency of large chips during inference: decoding requires a computation-to-data ratio of 1:1, whereas GPUs provide a 1000:1 ratio, leading to throughput loss. XPower's RPP architecture uses distributed small chips, enabling fine specialization within a single server. Furthermore, XPower's approach replaces expensive HBM technology with the more mature LPDDR, asserting that three elements are fundamentally necessary for AI inference: high capacity, sufficient bandwidth, and low cost. LPDDR technology meets all these requirements at a significantly lower cost than HBM. The configuration of 12 chips per card allows for detailed memory partitioning and parallel data processing between chips within a single server form factor. This distributed server architecture differentiates XPower from the NVIDIA Groq approach, which also uses distributed multi-chip designs, but on a much larger cluster scale. The inherent flexibility of the RPP architecture, balancing general and domain-specific computations, allows one chip to be used for deployment scenarios in the cloud, at the edge, and on the end device.
During WAIC, the company showcased its product line, ranging from the AE7100E chip to high-density server racks with 8 cards. The company's transition from edge computing to cloud is a strategic bet that the inference market—which XPower estimates to be ten times larger than the training market by total addressable value—will be captured not by the largest computational power of a single chip, but by the most cost-effective distributed systems. Although confirmation of this hypothesis will depend on real-world implementation results, the RPP architecture represents one of the most original domestic approaches to computing, challenging the consensus on large chips by asserting that the future of AI inference belongs not to single monolithic processors, but to clusters of coordinated small chips.
