TGViewer
Channel Public Channel
rtfm.co.ua | English

rtfm.co.ua | English

@rtfmcoua_en

rtfm.co.ua blog updates in English
Chat with us: https://t.me/rtfmco (UA/EN)
Open in Telegram
Subscribers
21
Photos
0
Videos
0
Links
45
Recent Posts 20 shown
Post #49 10
AI: LLMs, Agents, Work, and Us – Engineers. Personal Thoughts.

A very random text. Wasn’t planned at all, and there aren’t that many posts like this on this blog, but sometimes something “clicks in my head” (while there’s still something there to click) – and a thought takes shape that I want to express in a text like this, to save it. “Agentic development is…

https://rtfm.co.ua/en/ai-llms-agents-work-and-us-engineers-personal-thoughts/

#AI #personal_reflections #RTFM
RTFM: Linux, DevOps, and system administration | DevOps-engineering, and system administration. Cases from practice. AI: LLMs, Agents, Work, and Us – Engineers. Personal Thoughts. Personal reflections on working with AI agents - what they give us, what they take away, and how to keep thinking for yourself.
  • 🔥 1
Post #48 20
AI: LLM Spend Control with LiteLLM Budgets and OpenAI Limits

The main goal we had when introducing LiteLLM AI Gateway on the project (see LiteLLM: AI Gateway for LLMs – features overview) was access and cost control, because right now we can’t enable auto-recharge for our OpenAI or OpenRouter account: our startup is still in the development and experimentation stage, developers actively use AI agents…

https://rtfm.co.ua/en/ai-llm-spend-control-with-litellm-budgets-and-openai-limits/

#AI #Grafana #LiteLLM #LLM #VictoriaMetrics
RTFM: Linux, DevOps, and system administration | DevOps-engineering, and system administration. Cases from practice. AI: LLM Spend Control with LiteLLM Budgets and OpenAI Limits Setting up and monitoring LLM spends control with LiteLLM Budgets and OpenAI Limits- VictoriaMetrics, metrics, examples of Grafana dashboards and alerts.
Post #47 25
LiteLLM: Debugging AI Cost Monitoring with VictoriaMetrics

Had a pretty interesting case with monitoring LLM costs through LiteLLM when using multiple providers. What we have: - LiteLLM: AI Gateway, all our services work through it, it proxies requests to providers, generates metrics and traces, controls access, spending, etc. - OpenAI and OpenRouter: currently the two main providers LiteLLM sends requests to ◦…

https://rtfm.co.ua/en/litellm-debugging-ai-cost-monitoring-with-victoriametrics/

#AI #LiteLLM #LLM #OpenAI #VictoriaMetrics #VictoriaTraces
RTFM: Linux, DevOps, and system administration | DevOps-engineering, and system administration. Cases from practice. LiteLLM: Debugging AI Cost Monitoring with VictoriaMetrics An example of debugging OpenAI and OpenRouter costs using LiteLLM AI Gateway, VictoriaMetrics, and VictoriaTraces
Post #46 26
LiteLLM: Custom Callbacks and LLM Evaluations with Judge LLM

A quick recap of what we are doing on the project right now: we have a separate hardware server (hostname == “Matrix”), where we run our own self-hosted models with llama.cpp. In our Kubernetes cluster we have a LiteLLM AI Gateway for our clients – Backend API and other project services. Clients send their OpenAI/Anthropic/OpenRouter…

https://rtfm.co.ua/en/litellm-custom-callbacks-and-llm-evaluations-with-judge-llm/

#AI #LiteLLM #LLM #observability #Python #VictoriaMetrics #VictoriaTraces
RTFM: Linux, DevOps, and system administration | DevOps-engineering, and system administration. Cases from practice. LiteLLM: Custom Callbacks and LLM Evaluations with Judge LLM Traffic mirroring and LLM evaluations with LiteLLM and Judge LLM - score self-hosted LLM responses, metrics in VictoriaMetrics, traces in VictoriaTraces
Post #45 20
LiteLLM: Custom Callback for Traffic Mirroring and OTel Tracing to VictoriaTraces

We’re currently setting up our own hardware server where we want to run self-hosted models. But we can’t just switch client traffic to them right away – first we need to see how these self-hosted LLMs will actually perform. So the general idea for now is to keep sending traffic to the primary provider, OpenAI…

https://rtfm.co.ua/en/litellm-custom-callback-for-traffic-mirroring-and-otel-tracing-to-victoriatraces/

#AI #LiteLLM #LLM #OpenTelemetry #Python #VictoriaTraces
RTFM: Linux, DevOps, and system administration | DevOps-engineering, and system administration. Cases from practice. LiteLLM: Custom Callback for Traffic Mirroring and OTel Tracing to VictoriaTraces Writing your own LiteLLM Custom Callback in Python to implement traffic mirroring on a self-hosted LLM and configure OTel tracing with custom spans to VictoriaTraces
Post #44 25
LiteLLM: Traffic Mirroring, Batch Completions, and Traffic to Two Providers

We’re getting ready to launch a self-hosted LLM, and at the testing stage the general idea is to send client requests simultaneously both to the “default production model” like GPT-5.6 and to the model running on our own server. And after getting the responses, we’ll compare them with Phoenix or Opik, and gradually tune our…

https://rtfm.co.ua/en/litellm-traffic-mirroring-batch-completions-and-traffic-to-two-providers/

#AI #LiteLLM #observability #OpenTelemetry #VictoriaTraces
RTFM: Linux, DevOps, and system administration | DevOps-engineering, and system administration. Cases from practice. LiteLLM: Traffic Mirroring, Batch Completions, and Traffic to Two Providers A Practical Test of LiteLLM Traffic Mirroring and Batch Completions - Two Providers, Two Responses, and Unexpected Behavior in Logs and Traces
Post #43 19
llama.cpp: Metrics and Monitoring with VictoriaMetrics

We have a server where we’re going to run self-hosted LLMs. We spent quite a while choosing what exactly to use for running the models – vLLM, SGLang, or llama.cpp, and eventually settled on llama.cpp – at least for now. In the post NixOS: getting started, package installation, and system configuration I described installing Node…

https://rtfm.co.ua/en/llama-cpp-metrics-and-monitoring-with-victoriametrics/

#AI #llama_cpp #monitoring #VictoriaMetrics
RTFM: Linux, DevOps, and system administration | DevOps-engineering, and system administration. Cases from practice. llama.cpp: Metrics and Monitoring with VictoriaMetrics Configuring monitoring for llama.cpp and LLM with VictoriaMetrics - key metrics and details on working with llama.cpp and /metrics
Post #42 24
LiteLLM: Metrics, Traces, and Debugging exception_class=”ValueError”

A few days ago, I ran into an interesting situation with LiteLLM: on the one hand, the metrics showed a lot of errors “from the provider”, while on the other hand, the traces and alerts showed only a single error. I had to dig into it a bit and figure out some nuances of how…

https://rtfm.co.ua/en/litellm-metrics-traces-and-debugging-exception_classvalueerror/

#AI #LiteLLM #observability #VictoriaMetrics
RTFM: Linux, DevOps, and system administration | DevOps-engineering, and system administration. Cases from practice. LiteLLM: Metrics, Traces, and Debugging exception_class=”ValueError” Investigating the exception_class="ValueError" in LiteLLM with VictoriaMetrics, VictoriaTraces, and VictoriaLogs, and nuances of LiteLLM metrics and traces
Post #41 25
NixOS: Getting Started, Installing Packages, and Configuring the System

We got a new instance, a hardware server that will run our self-hosted LLMs. The server will run NixOS – not my choice, but the system looks interesting. I’ve been hearing about it for a long time, and now I have a great opportunity to get familiar with it. For now, my part is only…

https://rtfm.co.ua/en/nixos-getting-started-installing-packages-and-configuring-the-system/

#Amazon_Linux #Arch_Linux #monitoring #NixOS #ssh
RTFM: Linux, DevOps, and system administration | DevOps-engineering, and system administration. Cases from practice. NixOS: Getting Started, Installing Packages, and Configuring the System NixOS Getting Started - installing system and packages, systemd configuration, setting up a static IP address, and remote deployment via Flakes over SSH.
Post #40 28
LiteLLM: OpenRouter Integration and Fallbacks Configuration

OpenRouter recently announced a 50% discount on OpenAI, so we decided to give it a try. We’ll switch things over on LiteLLM, which already handles requests from all our services – so the switch should be pretty simple. Before OpenRouter, we’ll set up fallbacks straight to OpenAI in LiteLLM – for cases when requests to…

https://rtfm.co.ua/en/litellm-openrouter-integration-and-fallbacks-configuration/

#AI #LiteLLM
RTFM: Linux, DevOps, and system administration | DevOps-engineering, and system administration. Cases from practice. LiteLLM: OpenRouter Integration and Fallbacks Configuration Adding OpenRouter as a model provider in LiteLLM, configuring fallbacks to OpenAI on failure, and routing requests via tags or User-Agent.
Post #39 29
MongoDB: run in Kubernetes with MongoDB Operator and MongoDBCommunity CRD

Following up on the previous post Valkey: running in Kubernetes – Helm, monitoring, ACL – we set up Valkey/Redis there, now it’s time to add MongoDB. This is for our new service, which is still in the experimental/PoC phase, but will most likely go to production, so the setup is a bit “under-production” – we’re…

https://rtfm.co.ua/en/mongodb-run-in-kubernetes-with-mongodb-operator-and-mongodbcommunity-crd/

#Helm #Kubernetes #MongoDB
RTFM: Linux, DevOps, and system administration | DevOps-engineering, and system administration. Cases from practice. MongoDB: run in Kubernetes with MongoDB Operator and MongoDBCommunity CRD Example of deploying the MongoDB Operator in Kubernetes and creating a MongoDB Community replica set
Post #38 22
Valkey: Deployment in Kubernetes – Helm, Monitoring, ACL

We’re launching a new service for the project, and this service needs Redis and MongoDB. The “technical task” itself looks roughly like this: - Redis: task queue + some other stuff (e.g. we stream the agent response from LLM to Redis, and if the user is connected on a websocket – then we stream data…

https://rtfm.co.ua/en/valkey-deployment-in-kubernetes-helm-monitoring-acl/

#Helm #Kubernetes #Redis #Valkey
RTFM: Linux, DevOps, and system administration | DevOps-engineering, and system administration. Cases from practice. Valkey: Deployment in Kubernetes – Helm, Monitoring, ACL An example of deploying Valkey/Redis with master-slave replication in Kubernetes using Helm, monitoring, creating users, and an Access Control List
Post #37 31
LiteLLM: Monitoring with VictoriaMetrics – Alerts and Grafana

This is the second part of the LiteLLM monitoring series – in the previous one, we covered the general integration with VictoriaStack and looked at the metrics and traces we get from LiteLLM (see LiteLLM: metrics, traces, and integration with the VictoriaMetrics Stack). Now let’s move on to the practical part – what to monitor…

https://rtfm.co.ua/en/litellm-monitoring-with-victoriametrics-alerts-and-grafana/

#AI #LiteLLM #monitoring #VictoriaMetrics
RTFM: Linux, DevOps, and system administration | DevOps-engineering, and system administration. Cases from practice. LiteLLM: Monitoring with VictoriaMetrics – Alerts and Grafana Configuring Basic Monitoring for AI Gateway LiteLLM with VictoriaMetrics - Examples of Creating Alerts and Grafana Dashboards
Post #36 33
LiteLLM: Metrics, Traces, and VictoriaMetrics Stack Integration

Third part on running LiteLLM – AI Gateway or LLM Proxy, and finally we’re getting to monitoring. In the first part we got familiar with LiteLLM in general (see LiteLLM: AI Gateway for LLMs – overview of features), and in the second one we deployed it in Kubernetes and hooked up VictoriaTraces and VictoriaMetrics to…

https://rtfm.co.ua/en/litellm-metrics-traces-and-victoriametrics-stack-integration/

#AI #LiteLLM #monitoring #observability #VictoriaMetrics #VictoriaTraces
RTFM: Linux, DevOps, and system administration | DevOps-engineering, and system administration. Cases from practice. LiteLLM: Metrics, Traces, and VictoriaMetrics Stack Integration Configuring metrics and traces from LiteLLM to VictoriaMetrics and VictoriaTraces. Important LiteLLM settings, key metrics and traces, and their attributes.
Post #35 34
LiteLLM: AI Gateway on Kubernetes and Metrics in VictoriaMetrics

In the first part – LiteLLM: AI Gateway for LLMs – features overview we got familiar with what LiteLLM can do in general – now we can run it in Kubernetes and connect clients. At the same time we’ll check the integration with our existing monitoring stack – for now just metrics to VictoriaMetrics. Logs…

https://rtfm.co.ua/en/litellm-ai-gateway-on-kubernetes-and-metrics-in-victoriametrics/

#AI #AWS #Helm #Kubernetes #LiteLLM #VictoriaMetrics
RTFM: Linux, DevOps, and system administration | DevOps-engineering, and system administration. Cases from practice. LiteLLM: AI Gateway on Kubernetes and Metrics in VictoriaMetrics How to deploy LiteLLM AI Gateway on Kubernetes with Helm, AWS RDS PostgreSQL, and Redis, External Secrets Operator, and collect metrics to VictoriaMetrics
Post #34 33
Claude Code: Monitoring with OpenTelemetry and VictoriaMetrics

While working on LiteLLM (see LiteLLM: AI Gateway for LLMs – features overview), I had an idea: besides services like our Backend API, why not also monitor the Claude Code developers? Just out of curiosity – to see what’s going on there in general and how everyone uses our Anthropic Organization, because a lot of…

https://rtfm.co.ua/en/claude-code-monitoring-with-opentelemetry-and-victoriametrics/

#AI #Claude_Code #monitoring #VictoriaMetrics
RTFM: Linux, DevOps, and system administration | DevOps-engineering, and system administration. Cases from practice. Claude Code: Monitoring with OpenTelemetry and VictoriaMetrics Claude Code monitoring for Organization and managed settings - OpenTelemetry metrics, traces, and logs with VictoriaMetrics, VictoriaTraces, and VictoriaLogs.
Post #33 33
LiteLLM: AI Gateway for LLMs – Features Overview

In the previous posts on OpenTelemetry and VictoriaTraces (see OpenTelemetry: OTel Collectors in Kubernetes and integration with the VictoriaMetrics stack and VictoriaTraces: Tracing, Observability and OpenTelemetry) we covered the general concepts of what observability is and how to work with traces. But this topic actually came up on the project when we realized that using…

https://rtfm.co.ua/en/litellm-ai-gateway-for-llms-features-overview/

#AI #LiteLLM #LLM #security #VictoriaTraces
RTFM: Linux, DevOps, and system administration | DevOps-engineering, and system administration. Cases from practice. LiteLLM: AI Gateway for LLMs – Features Overview Getting Started with LiteLLM AI Gateway - Running in Docker, OpenTelemetry traces in VictoriaTraces, access and limits via Virtual Keys, Teams, and budgets
Post #32 30
MikroTik: User Management, access permissions, and SSH

Time to finally write up MikroTik and Users Management – this one’s been sitting in drafts for ages, and while I’m at it, I’ll also set up SSH key authentication. Let’s walk through the main concepts and settings of Authentication, Authorization, Accounting in MikroTik – groups, policies, and users. What we have and what needs…

https://rtfm.co.ua/en/mikrotik-user-management-access-permissions-and-ssh/

#MikroTik #security #ssh
RTFM: Linux, DevOps, and system administration | DevOps-engineering, and system administration. Cases from practice. MikroTik: User Management, access permissions, and SSH How to configure access in MikroTik RouterOS - creating Groups and Policies, adding Users, restricting access by IP, and setting up SSH key authentication.
Post #31 22
VictoriaTraces: Recording Rules, Metrics, and Alerts from Trace Spans

VictoriaTraces – just like VictoriaLogs – supports Recording Rules (see VictoriaMetrics: Recording Rules for AWS Load Balancer logs) for traces, because traces are essentially the same logs, just structured differently. And since we have Recording Rules – we can build metrics out of logs for alerts and Grafana dashboards. Although this actually isn’t the best…

https://rtfm.co.ua/en/victoriatraces-recording-rules-metrics-and-alerts-from-trace-spans/

#Grafana #OpenTelemetry #VictoriaMetrics #VictoriaTraces
RTFM: Linux, DevOps, and system administration | DevOps-engineering, and system administration. Cases from practice. VictoriaTraces: Recording Rules, Metrics, and Alerts from Trace Spans Example of creating Recording Rules for metrics and alerts from OpenTelemetry traces in VictoriaTraces - latency and errors in a Backend API, AWS ALB, and PostgreSQL
Post #30 18
Arch Linux: a DNS Mystery – VPN, systemd-resolved, and Unbound

I’d been wrestling with the problem of accessing AWS EKS from the office for a long time – finally lost my patience and figured it out 🙂 Here’s the problem: there’s an AWS EKS cluster with both Public and Private endpoints for the API. Working from my office laptop, sometimes requests to it go through…

https://rtfm.co.ua/en/arch-linux-a-dns-mystery-–-vpn-systemd-resolved-and-unbound/

#Arch_Linux #AWS #DNS #Networking #Unbound #VPN
RTFM: Linux, DevOps, and system administration | DevOps-engineering, and system administration. Cases from practice. Arch Linux: a DNS Mystery – VPN, systemd-resolved, and Unbound Arch Linux, AWS EKS API endpoint and random DNS - how VPNs, routing tables, and systemd-resolved can break DNS resolution, and a fix with Unbound on Arch Linux
Older posts →

About this channel

How can I read @rtfmcoua_en without a Telegram account?
TGViewer shows the public web preview Telegram publishes for rtfm.co.ua | English: recent posts, photos, videos and the subscriber count, with no app, login or account.
How many subscribers does rtfm.co.ua | English have?
rtfm.co.ua | English (@rtfmcoua_en) has 21 subscribers on Telegram, refreshed roughly every 30 minutes.
Does rtfm.co.ua | English know I viewed it here?
No. Public channel previews carry no viewer identity, and TGViewer has no accounts or tracking of what you look up.
Threads Profile ViewerView any public Threads profile without an account.Open ThreadLook →Writing with AI? Make it sound human.Metric37 rewrites AI drafts so they read naturally. Free AI detector, 1,500 words free.Try Metric37 →