🚀 HuggingFace Releases Datasets for Pre-Training LLM in Code Generation

Following the success of OlympicCoder-32B, which beat Sonnet 3.7 in LiveCodeBench and IOI 2024, HuggingFace has released a rich dataset for pre-training and fine-tuning LLM in programming tasks.

✅Stack-Edu (125 billion tokens) – educational code in 15 programming languages, filtered from The Stack v2
✅GitHub Issues (11 billion tokens) – data from discussions and bug reports on GitHub
✅ CodeForces problems (10K tasks) – a unique set of CodeForces problems, 3K of which were not used in DeepMind training
✅ CodeForces problems DeepSeek-R1 (8.69 GB) – filtered traces of CodeForces solutions
✅ International Olympiad in Informatics: Problem statements dataset (2020 - 2024) - a unique set of programming Olympiad tasks, divided into subtasks so that each query corresponds to a solution to these subtasks
✅ International Olympiad in Informatics: Problem - DeepSeek-R1 CoT dataset (2020 - 2023) - 11 thousand traces of reasoning performed by DeepSeek-R1 during the solution of programming Olympiad tasks

💡 What to use it for?
🔹 LLM pre-training for code generation
🔹 Developing AI assistants for programmers
🔹 Improving solutions in computer olympiads
🔹 Creating ML models for code analysis