The GigaChat team has released GigaChat 3.5 Ultra as open source—a new 432B model under the MIT license. This is the first open-source hybrid of GatedDeltaNet and MLA scaled to hundreds of billions of parameters, featuring a proprietary training recipe we refined through more than 1,500 experiments. The model has grown in terms of code, mathematics, agent scenarios, and application domains—yet it’s 40% smaller than GigaChat 3.1 Ultra.
What’s inside:
🔘A proprietary hybrid MLA + Gated DeltaNet architecture with a dedicated stabilization framework, without which this hybrid setup would not train reliably at this scale;
🔘 Gated Attention: the model can locally down-weight overly strong signals from the attention layer;
🔘GatedNorm: normalization with an explicit gate that controls signal magnitude across features;
🔘Approximately 4x lower KV cache per token: with the same memory budget, the model can support 2.14x longer context and deliver a 20% throughput increase under load;
🔘Two MTP heads, enabling up to 2.2x faster generation;
🔘FP8 across all training stages with no quality degradation compared with bf16, enabled by custom Triton and CUDA kernels;
🔘A new online RL stage after SFT and DPO.
Results:
🔘 GigaChat-3.5-Ultra-Base outperforms DeepSeek V3.2 Exp Base and DeepSeek V4 Flash Base on average across a set of general, math, and code benchmarks:
🔘 GigaChat-3.5-Ultra-Instruct is comparable to DeepSeek V3.2 in terms of average score, despite having half the size;
🔘 According to the MiniMax-M2.7 LLM judge, the average win rate against GigaChat 3.1 Ultra is 75.9%, and against GPT-5 is 68.7%.
The entire stack — data (our own LLM-filtered Common Crawl, 600+ programming languages in the code), architecture, training methodology, and infrastructure — was built end-to-end by GigaChat team.
➡️ HuggingFace