😎💡FineMath: A New Math Dataset by Hugging Face

Hugging Face has released FineMath, a comprehensive dataset for training models on mathematical content. It was constructed using CommonCrawl, a classifier trained on LLama-3.1-70B-Instruct annotations, and a thorough data filtering process.

Compared to OpenWebMath and InfiMM, FineMath shows more consistent accuracy improvements as the dataset size increases, thanks to its high quality and diverse content.

A project utilizing FineMath for training LLMs in math assistance is already live — explore the GitHub repository.