StepFun has introduced the preliminary version of the Step 5 model, which is a base model featuring a sparse Mixture-of-Experts (MoE) architecture and approximately 600 billion total parameters, with about 27 billion activated per token. This release targets long-horizon agent workloads, including AI-assisted code development, software engineering, financial analysis, and professional knowledge work. The model supports a one-million-token context window and accepts both text and image inputs. StepFun announced that API access is already open, and the model weights are scheduled to be made publicly available on October 15th.
Instead of expanding the network, Step 5 Preview utilizes a narrow, deep Transformer architecture consisting of 92 layers. The company asserts that deeper stacks provide longer information pathways for implicit multi-step reasoning during long prefill, when agents perform searches, run code, and process tool usage results. To maintain practicality in the million-token sessions, the model incorporates Sparse Grouped-Query Attention with token block merging. According to StepFun, this reduces the cost of indexer selection and top-k by roughly one-eighth compared to a denser base model while consolidating overlapping adjacent selections.
The training process emphasizes on-policy long-horizon reinforcement learning, as well as bit-level alignment of training and inference in MoE routing. Furthermore, techniques such as load-aware scheduling, MTP-3 speculative decoding, FP8 MoE, and KV cache offloading are employed. StepFun reports more than a threefold acceleration of the end-to-end RL process for the long horizon and a sample registry loss metric below one percent. This model is also used within a human-managed data pipeline that generates verifiable complex tasks at scale in the millions, covering science, software development, and machine learning research.
Regarding AI analysis, Step 5 Preview scores around 44 points on an intelligence index that StepFun ranks among the best open-weights models in this rating system. The API cost, according to published pricing, is approximately $1 per million input tokens and $2.7 per million output tokens, with an output speed of about 100 tokens per second. The model's primary focus is on agent architecture and efficiency, distinguishing it from previous releases like Step 3.5 Flash and Step 3.7 Flash, emphasizing depth, sparse long context, and reliable multi-stage tool use before the weights become available in October.