aaargh i should've wrote this paper!! it was intuitively obvious to me but then life happens >_<

tldr: LLM sampler is such a powerful prior that with the right sampler (MCMC, in this case), you can even use base models as reasoning models.
without supervised fine-tuning or RL.

this was completely ignored by ppl pilled with the Bitter Lesson mantra, but yes there still is a space for the right priors added or designed by hand!

obviously sampling with mcmc is very costly but you should compare the overall model feedback loop time that includes the posttrain, not just the sampling time

if eg topK sampling is assembly and Mirostat is COBOL (?) then MCMC sampling is like a Python in the space of samplers
https://aakaran.github.io/reasoning_with_sampling/