💡 SmolTalk: a synthetic English-language dataset for LLM education

SmolTalk is a synthetic dataset from HuggingFace designed for teacher-led LLM learning. It consists of 2 million rows and was used to develop SmolLM2-Instruct models.

🔥Dataset includes both new and existing datasets

😎New datasets:

✅Smol-Magpie-Ultra (400k rows).
✅Smol-constraints (36,000 rows)
✅Smol-rewrite (50 thousand lines)
✅Smol-summarize (101 thousand lines)

⚡️Older datasets:

✅OpenHermes2.5 (100 thousand lines)
✅MetaMathQA (50 thousand lines)
✅NuminaMath-CoT (1120 thousand lines)
✅Self-Oss-Starcoder2-Instruct (1120 thousand lines)
✅SystemChats2.0 (30 thou. lines)
✅LongAlign (less than 16 thousand tokens)
✅Everyday-conversations (50 thousand lines)
✅APIGen-Function-Calling (80k lines)
✅Explore-Instruct-Rewriting (30k lines)

📚Training results:
SmolTalk showed significant improvements in model performance, especially in the tasks of math, programming, and following system prompts. SmolTalk training gave better results on IFEval, BBH, GS8Mk and MATH labels, including when training Mistral-7B.