Modular provides MAX platform - it is MAX inference backend (engine) and MAX inference server (MAX Serve).
Just look at this:
https://builds.modular.com/models/DeepSeek-R1-Distill-Llama/8B-Q6_K
https://builds.modular.com/models/Llama-3.3-Instruct/70B?tab=deploy
In terms of deployment it is fantastic! Just one (relatively) tiny container!
And in terms of programming - GPU programming and acceleration without CUDA, using Mojo language (statically LLVM compiled), which has capabilities of Rust (static memory safety), LLVM MLIR (Multi-Level Intermediate Representation) byte code compilation for amazing low level code optimization and acceleration, syntax of Python and Mojo integrates (embrace) the whole Python ecosystem. I'm playing with Mojo for quite a while already (and it is best of both worlds - Rust and Python), but MAX just used recently. And Llama.cpp not even in comparison with MAX!
#Mojo
#MAX
#AI
#AGI
Post #1583
438
Technologique Amazing things has been released by Modular development team (Mojo language and Max inference backend): https://www.modular.com/blog/max-25-2-unleash-the-power-of-your-h200s-without-cuda #Mojo #MAX #AI #AGI
