Everyone knows about LLM aka Large Language model.

Now we will talk about SLM aka Small Language model

As their name implies, SLMs are smaller in scale and scope than large language models.

Some examples of SLM are
- Phi 3.5
- tiny Llama
- mobile Llama
- Gemma2

SLMs can be trained using two main techniques:

Knowledge distillation: A smaller model learns from a larger, already-trained model

Pruning: Extra bits that aren't needed are removed to make the model faster and leaner

Here are some characteristics of SLMs:

Smaller in size: SLMs have fewer parameters than LLMs, often in the tens to hundreds of millions, compared to billions in LLMs.

More efficient: SLMs are more computationally efficient and can run on less powerful hardware.

Faster training: SLMs can be trained and developed faster than LLMs.

Specialized: SLMs are trained on curated data sources and can be specialized in specific tasks.

Fine-tunable: SLMs can be fine-tuned to do exactly what is needed for a specific task.

Cost-effective: SLMs can be more cost-effective than LLMs, making them a good option for integrating intelligent features when resources are limited.