DevOps & SRE notes

I found a good example of why autoscaling based only on CPU utilization can cause an outage.

About a week ago, Twingate had an incident that affected us as a client. They've published a postmortem, and it's a good example of why CPU isn't a good metric to rely on when autoscaling your services.

The incident was triggered by elevated network latency affecting communication paths used by the Authorization service. As requests took longer to complete, individual service instances were able to process fewer requests than normal.

This reduction in throughput exposed a limitation in our auto-scaling configuration, which primarily relied on CPU utilization to determine service capacity requirements.

So, from the CPU utilization perspective, everything was OK, but the number of processed requests decreased.

https://status.twingate.com/incidents/49qvqk7swjpq

Networking within container orchestration can often seem like a black box to developers. This explanation aims to demystify Kubernetes CNI providers and how they manage connectivity.
https://medium.com/@csinclair11/demystifying-kubernetes-cni-providers-5ed79569c797

The article details how to implement production-grade distributed tracing for complex multi-agent AI workflows using OpenTelemetry.

https://developers.redhat.com/articles/2026/04/06/distributed-tracing-agentic-workflows-opentelemetry#

Many organizations are looking for more efficient logging solutions than the traditional stack. This comparison highlights a modern alternative to ELK that aims to reduce complexity and resource usage.
https://osuite.io/articles/modern-alternative-to-elk

kro | Kube Resource Orchestrator
https://github.com/kubernetes-sigs/kro

This informative post details a clever method for securing Grafana dashboards when using Google Cloud Identity-Aware Proxy. You will learn how to seamlessly integrate these two powerful technologies for enhanced access control.
https://www.vidbregar.com/blog/grafana-gcp-iap

Managing expenses in the cloud requires a strategic approach beyond just looking at bills. A senior engineer shares valuable insight into optimizing costs effectively in this detailed read.
https://medium.com/@razkevich8/cloud-cost-optimization-a-senior-engineers-guide-d49ed4606de1

A popular & widely deployed Open Source Container Native Storage platform for Stateful Persistent Applications on Kubernetes.

https://github.com/openebs/openebs

The observability market is shifting from volume-based data ingestion to a value-driven model due to the unsustainable costs of scaling cloud-native and AI workloads. Driven by innovations like Chronosphere’s "Logs 2.0" and its subsequent acquisition by Palo Alto Networks, the industry is prioritizing "signal discipline"—retaining only actionable telemetry—and integrating observability directly into broader AI and security platforms.

https://siliconangle.com/2026/02/05/observability-cost-ai-scale-chronosphere-opensourcesummit/

Claude Code gave me three "tickets" for a free week. You can grab them using this link: https://claude.ai/referral/NXtyf-cgbQ

Uber engineered an automated approach to migrate its massive Java monorepo (over 600,000 tests, 15 million lines of code) from the deprecated JUnit 4 to JUnit 5. Facing challenges like the lack of native JUnit 5 support in their Bazel build system and custom test configurations, they successfully migrated over 75,000 test classes and 1.25 million lines of code in just four months without disrupting developer workflows.

https://www.uber.com/us/en/blog/junit-migration/

The article explains that while Kubernetes excels at scheduling and isolating workloads, it lacks the context to secure Large Language Models (LLMs), which process untrusted natural language inputs. Highlighting four key risks from the OWASP Top 10 for LLMs, the author argues that security controls shouldn't live within the model runtime (like Ollama). Instead, organizations need a dedicated, LLM-aware policy layer (such as LiteLLM, Kong AI Gateway, or Portkey) in front of the model to enforce validation, filtering, and authorization.

https://www.cncf.io/blog/2026/03/30/llms-on-kubernetes-part-1-understanding-the-threat-model/

the Dirty Frag vulnerability class, first discovered and reported by Hyunwoo Kim (@v4bel), which can obtain root privileges on major Linux distributions by chaining the xfrm-ESP Page-Cache Write vulnerability and the RxRPC Page-Cache Write vulnerability.

https://github.com/V4bel/dirtyfrag

Bulk port forwarding Kubernetes services for local development.
https://github.com/txn2/kubefwd

The article features an interview with Landon Clipp, who built a multi-tenant GPU-based CaaS platform.
- Bypassing the NVIDIA GPU Operator
- Why gVisor Fails for GPUs
- VM Boot Delays
- Firmware and Memory Security
- Ideal Workload

https://kube.fm/gpu-containers-as-a-service-landon

Kubernetes Goat is a "Vulnerable by Design" cluster environment to learn and practice Kubernetes security using an interactive hands-on playground 🚀

https://github.com/madhuakula/kubernetes-goat

Any user with Argo CD application get permissions can extract real Kubernetes Secret values including service account tokens, TLS certificates, database credentials, and API keys. On Applications where IncludeMutationWebhook=true is already set, exploitation requires only read-only Argo CD access.

https://github.com/argoproj/argo-cd/security/advisories/GHSA-3v3m-wc6v-x4x3

🚀2.3x faster than MinIO for 4KB object payloads. RustFS is an open-source, S3-compatible high-performance object storage system supporting migration and coexistence with other S3-compatible platforms such as MinIO and Ceph.

https://github.com/rustfs/rustfs

⚰️ Minio was archived on Apr 25, 2026
https://github.com/minio/minio

It's time to update your kernel

An unprivileged local user can write 4 controlled bytes into the page cache of any readable file on a Linux system, and use that to gain root.

https://copy.fail/