Thinking Machines presents a mathod that could change how language models are trained. It is called on-policy distillation which teaches AI not just to copy, but to think and analyze its mistakes.
🟢 How it works?
A large teacher model shows answers, and a smaller student model memorizes them. It’s like learning by rote — fast, but without understanding the essence.
In the new approach, it’s different. The student solves tasks on its own, and the teacher evaluates and guides explaining where the logic fails and how to improve reasoning. Thus, the smaller model inherits not only knowledge but also the way of thinking of the larger model.
🟠 What the results showed?
Experiments were conducted on tasks involving mathematical and logical reasoning, where it’s important not just to give the correct answer but to build a chain of steps.
The results are impressive:
The student model after training with on-policy distillation showed almost the same accuracy as the much larger teacher model.
At the same time, computational costs were reduced several times, making the model noticeably more efficient and cheaper.
Moreover, the student became better at understanding its own mistakes, which increased robustness and reliability when solving new, unfamiliar tasks.
🟢 Why this matters?
On-policy distillation solves a key problem of traditional methods — lack of adaptability.
The model now learns from its own steps, like a human — experimenting, making mistakes, correcting behavior, and growing.
The uniqueness of the approach lies in the balance between the quality of RL and the economy of KD. This is a real scheme where a small model learns “in the field” (reacting to its own actions) but without expensive RL runs and complex reward models.
This is not a new training method, but a new engineering formula that allows cheaper “teaching” of compact models that behave like large ones.
This opens the way to creating compact next-generation LLMs that reason almost like top models but cost much less.
Such models can be run on edge devices, autonomous agents, and local services where speed, privacy, and energy efficiency are important.
#ThinkingMachines #LLM #ML
🤖 Data Science, ML & Big Data with @DataXplore