Microsoft has really turned the tables 🤯

They've long since released the open-source bitnet.cpp - a framework for inference of 1-bit LLMs.

It allows you to run models with 100B parameters directly on the local CPU, without any GPUs.

- inference is 6.17 times faster
- energy consumption on the CPU is 82.2% lower

And yes, it's 100% open source.

••••••••••••••••••••••••••••••••••••••••••••••••••••
🤖 Data Science, ML & Big Data with @DataXplore