🤖 Liquid AI's Liquid Neural Networks Challenge Pure Transformers

👤 Ramin Hasani - Co-founder and CEO of Liquid AI (developing efficient non-transformer AI models)
🎤 Swyx (Shawn Wang) - Co-founder of Latent Space (AI engineering media and VC fund), Richard - Editor-in-Chief of Latent Space

📺 Latent Space – 👥 183K subscribers
⏱ 1h 10m
🗓 18.09.26 (3d ago)
👁 5K

📝 Ramin Hasani explains how the nervous system of a 302-neuron worm inspired LFMs, why the future of AI belongs to on-device edge computing outside data centers, and how architecture meta-search slashes inference costs.

━━━━━━━━━━━━━━━

💡 KEY TAKEAWAYS

1️⃣ Liquid architecture is inspired by a 302-neuron worm
The C. elegans neural network controls 95 muscles better than traditional robotics systems. Its neurons do not spike but operate continuously in time, forming the foundation for differentiable liquid neural networks.

2️⃣ Scaling non-linear models breaks GPU parallelism
Recurrent dynamical systems deliver sub-linear compute complexity but run into modern hardware constraints. Scaling them requires linearizing dynamics via SSMs (State Space Models), trading off some expressivity.

3️⃣ Liquid AI moved away from betting on a single architecture
The startup built a hybrid architecture meta-search system (STAR framework). The algorithm automatically blends convolutions, attention layers, and recurrent blocks for specific chips without sacrificing generation quality.

4️⃣ The LFM-2 model is 80% 1D convolutions
Instead of pure attention, the second generation of Liquid Foundation Models relies on double gated convolutions and only 20% Group Query Attention. This optimizes performance on standard CPUs.

5️⃣ On-device local inference slashes business costs
Routing 90% of requests to the cloud is economically unviable at billion-user scale. Liquid deployed 600 MB multimodal models for Mercedes-Benz cars running on $100 chips.

6️⃣ Shopify processes over 1 billion requests per month via LFMs
The e-commerce platform integrated Liquid's private instances into the Shop app. Engineers rely on these models for low latency and consistent quality retention during streaming generation.

7️⃣ The scale paradox: complex loops only benefit small models
In small models (under 20B parameters), structural inductive biases and feedback loops boost quality. At massive parameter scale, simple, unbiased architectures perform best.

💬 «Easy problems, everyone else will go and solve. If the problem is hard, that makes it much more attractive for building a business.»

🎯 WHAT TO DO WITH THIS
To cut infrastructure costs, stop routing all traffic through heavy cloud LLMs. Test hybrid architectures combining convolutions and SSM mechanisms on local edge device silicon.

💵 0.11$

@icereadspodcasts