Yesterday we did our live stream using c3-llama-cpp (https://github.com/comput3ai/c3-llamacpp). We've now replaced that with the final boss of high performance LLM APIs (https://github.com/comput3ai/c3-vllm).

How much of a difference did it make? 200-350 token/s yesterday. Now we're getting 2000-3000 token/s. that's roughly 8x. It's insanely fast. We'll be beta testing this next week, and properly launching it by the end of next week.

ABC - Always be cooking.

$COM J3NrhzUeKBSA3tJQjNq77zqpWJNz3FS9TrX7H7SLKcom