The article explains that while Kubernetes excels at scheduling and isolating workloads, it lacks the context to secure Large Language Models (LLMs), which process untrusted natural language inputs. Highlighting four key risks from the OWASP Top 10 for LLMs, the author argues that security controls shouldn't live within the model runtime (like Ollama). Instead, organizations need a dedicated, LLM-aware policy layer (such as LiteLLM, Kong AI Gateway, or Portkey) in front of the model to enforce validation, filtering, and authorization.

https://www.cncf.io/blog/2026/03/30/llms-on-kubernetes-part-1-understanding-the-threat-model/