Nemotron-Cascade 2 - world's second open language model that won gold medals in three competitions in 2025: IMO, IOI, and the ICPC finals.
➡️ Why it matters?
Before it, only DeepSeek-V3.2-Speciale, a model with 671B parameters and 37B active ones, had achieved this. Nemotron-Cascade 2 has 20 times fewer parameters: 30B total, 3B active.
☞ At IMO 2025, the model solved 5 out of 6 tasks and scored 35 out of 42 points.
☞ At IOI - 439 out of 600, corresponding to gold.
☞ At ICPC World Finals, 10 out of 12 tasks were solved, ranking 4th in the gold zone. Eight ICPC tasks were solved in less than 100 attempts.
➜ Architecture
MoExperts based on Nemotron-3-Nano-30B-A3B-Base. The difference from the first version is an expanded Cascade RL, where reinforcement learning takes place sequentially across domains: first following instructions, then multi-domain RL, then distillation, RLHF, working with long context, code, and finally software engineering tasks.
➜ Multi-Domain On-Policy Distillation (MOPD)
At each stage of Cascade RL, the best intermediate checkpoint for a specific domain is selected and used as a teacher. Since all teachers come from a single SFT initialization, they share the tokenizer and vocabulary, which simplifies distillation.
MOPD works at the level of individual tokens, not sequences, and converges faster than classic GRPO: on AIME 2025, it reaches the teacher's level in 30 optimization steps.
➜ TESTS
On benchmarks, the model outperforms Qwen3.5-35B-A3B and the larger Nemotron-3-Super-120B-A12B in math, code, and following instructions.
☞ LiveCodeBench v6: 88.4 (Qwen3.5 - 74.6);
☞ ArenaHard v2: 83.5 vs. Qwen3.5's 65.4;
☞ IFBench: 82.9 vs. 70.2;
☞ On Codeforces, the model scored a rating of 2345 with TIR (on par with models with 300B+ parameters).
➜ WEAKNESSES
Tasks requiring deep knowledge (MMLU-Pro, GPQA-Diamond) and agent scenarios (BFCL v4, τ²-Bench, SWE Verified). Here, Qwen3.5 is still ahead.
NVIDIA promises to improve the science-intensive pretraining and agent RL in future versions of Nemotron-Cascade.
Licensing: NVIDIA Open Model License: Model, SFT dataset, RL data, Technical report
#AI #ML #LLM #Nemotron #Cascade2 #NVIDIA
••••••••••••••••••••••••••••••••••••••
🤖 Data & ML | @DataXplore