LiteLLM: Custom Callback for Traffic Mirroring and OTel Tracing to VictoriaTraces

We’re currently setting up our own hardware server where we want to run self-hosted models. But we can’t just switch client traffic to them right away – first we need to see how these self-hosted LLMs will actually perform. So the general idea for now is to keep sending traffic to the primary provider, OpenAI…

https://rtfm.co.ua/en/litellm-custom-callback-for-traffic-mirroring-and-otel-tracing-to-victoriatraces/

#AI #LiteLLM #LLM #OpenTelemetry #Python #VictoriaTraces