goinfer Runs Local GGUF and Safetensors Models in Pure Go

goinfer is a cgo-free Go runtime for running GGUF and safetensors models as a single static binary. It includes constrained JSON generation from Go structs and an OpenAI-compatible server, avoiding Python and CUDA setup for local deployments.

🔗 github.com

#Go #localLLM #inference