Step 5 (Jieyue) has open-sourced its 600-billion-weight model, featuring a 92-layer deep architecture that reduces index overhead for millions of tokens to one-eighth.

Beating AI News Flash: StepFun has open-sourced the Step 5 Preview model weights, with FP8 and BF16 versions now available on Hugging Face and ModelScope, while supporting inference code has also been released simultaneously. The Step 5 Preview model has a total of 600 billion parameters, with only approximately 27 billion activated per generation, and supports a 1-million-token context window. A newly released technical blog reveals the model adopts a 92-layer "narrow and deep" architecture, al

@tonmelonfest