A family of multilingual SLMs with 3 billion parameters and an 8K context window, which supports over 70 languages.
Touted as a worthy candidate for local translators, chatbots and offline educational tools. If you need it to be fast, local, and translate Swahili or Khmer better than Llama - this is it.
🟢 Why it matters?
☞ highlight of the release is in data engineering.
Tiny Aya was trained on 6 trillion tokens, and the problem of insufficient data for rare languages was solved through synthesis from teacher models (their own Command R + DeepSeek-V3).
Instead of training one model on everything at once, they divided the data into language clusters (Europe, Asia, Africa, etc.) and fine-tuned individual branches, after which they merged these regional checkpoints into the global Tiny Aya Global model.
☞ composition of the family
Tiny Aya Global: A universal checkpoint for all languages.
Tiny Aya Earth: Africa and West Asia.
Tiny Aya Fire: South Asia.
Tiny Aya Water: The Asia-Pacific region and Europe. We're here
GGUF: There's a 4, 8, and 16-bit version for each version.
iOS and Android: The models are available in PocketPal
☞ Test results
The global version beats Gemma 3-4B in 46 out of 61 languages on the WMT24++ benchmark.
On the iPhone 17 Pro, it outputs 32 tokens/sec, and on the old iPhone 13, it outputs about 10 tokens/sec in Q4_k_m quantization.
The highest security score (91.1%) among competitors (Qwen3-4B, Ministral-3-3B).
☞ A touch of realism
This is a 3B model. In complex tasks, it's obviously worse or somewhere near its peers, so don't expect miracles.
Despite the claimed diversity, English occupies the lion's share of the dataset in all clusters.
With strong compression (below Q4), the quality starts to suffer noticeably, especially in rare languages.
Blog, HF, Paper, Demo •#AI #ML #SLM #TinyAya #Cohere
••••••••••••••••••••••••••••••••••••••••••••••
🤖 Data & ML | @DataXplore