CatOps

So, that's for AI in the companies, but what about AI in the wild i.e. in open source?

We have cases like curl, that had to take down their bug bounty program due to the influx of slop bug reports. Yet, the industry adapts.

Here's a study by Redmonk on the stance of various foundations and standalone open source projects on AI, including their major concerns, and openness to AI-generated contributions.

#ai #open_source

Continuing with our AI week.

AI in SRE: What's Actually Coming in 2026 is telling a story of AI coming for help with incident response.

The article suggests trying an AI tool for real investigation or data collection for postmortems. To clarify this, in my experience, you don’t need to have a dedicated tool, a general purpose AI agent with some harness (skills and scripts) would do. You should try it! AI does the job of data gathering incredibly well. Yet, the results are indeed not perfect.

Another good point in this article is data quality. AI results are as good as context you provide. I witnessed two prominent failure modes so far:

1. Inference on incomplete data: a person with limited access (typically a developer) asks their agent to investigate an alert. The agent comes to some conclusion. At the same time, a person with elevated access (typically a systems engineer) asks their agent to investigate the same alert and gets a different result, likely because some data is only available via kubectl events, etc. The fix for that is not to allow everyone to do everything, the fix is to revisit your observability pipelines and ensure that you ship all the relevant data, which is easier said than done.
2. Agent that cries "wolves": if you have a pollutant in your logs, or simply an event that happens very often, agents like to correlate it with everything. If your clusters are elastic, an agent could blame node count fluctuations for every error. The problem here is that once node count fluctuation actually causes a problem, you will be the one to ignore this hint from an agent, because it suggests it every single time.

If you are ready to share more AI failure modes specifically related to SRE in Ukrainian, welcome to our chat.

#ai #sre

​​Harness engineering for coding agent users is a new guest article in Martin Fowler's blog that summarizes approaches to improve AI output and make it more manageable.

If you're actively using AI agents day-to-day, things described in this article won't be news to you, but it helps to structure one's thoughts.

#ai

I will post AI-related articles this week, because why not?

The first one is from Charity Majors called AI demands more engineering discipline. Not less, in which she follows up on her another article.

This one is on technical aspects of moving to the disposable code. It also has a lot of links to other articles, which is also cool.

#ai

​​For today's Donations Monday, I'd like to share with you a fundraiser that our friends at DOU started for the 2nd separate corps of the National Guard of Ukraine «Хартія». The goal of this fundraiser is to buy heavy bomber drones "Vampire" for the Kupiansk direction.

Monobank jar: https://send.monobank.ua/jar/26mrQPQ3PZ

#donations #Ukraine

A new issue of the CatOps Digest is here!

https://newsletter.catops.dev/p/catops-digest-2026-06-13

#digest #newsletter

A Reddit thread with some useful tools for Kubernetes and kubectl plugins0.

Some things there are well-known, but you may find some new interesting things there. I did :)

#kubernetes

​​How much do amd64 microarchitecture levels help in Go? is a benchmarking article that shows the compute time improvements you can get if you'd build your apps for modern x64 processors only. You likely use modern processors already and do not plan to run your apps on the decade old hardware.

Still, it's important to remember that while such articles are nice; your real applications probably don't just calculate bit vectors all day. It's much more likely your real performance bottleneck is I/O and not the fact that your apps are built with the support for old hardware. Still, you can get some easy wins here by just adding a compilation flag, if you're using Go.

#performance #go #programming

An explainer for the Backend-for-Frontend pattern. The article provides some high-level overview of what it is, and when to use it.

#architecture #design

​​For today's Donations Monday, I'd like to share with you requisites of a friend of mine, who volunteers for AFU since the beginning of the full-scale invasion.

Here's a page with all the possible ways to donate. You can also find links to the current goals, and reports for previous fundraises there.

Here's their Monobank jar that supports Apple Pay, if you'd better have a direct link:

https://send.monobank.ua/jar/BQjWbpver

#donations #Ukraine

​​How Do You Fit a Trillion-Parameter Model Into a Kubernetes Cluster? is an interesting article about how one should change their perspective when reasoning about running LLMs in Kubernetes compared to usual web apps.

It’s an interesting read, especially, if you don’t work with this stuff every day. The biggest takeaway here is that in the case of models, a “replica” doesn’t mean a pod in most of the cases, it’s a distributed system on its own that should behave as one. This article also explains, how exactly things are distributed within a replica, and what are the low level system parameters to pay attention to.

#kubernetes #ai #llm

​​A case study from Amazon, how science solves actual engineering problems that later translate in money savings (likely millions on the Amazon scale).

How a Slack shout-out, a dusted-off academic theory, and a spaghetti monster led an AWS team to crack an elusive code—and deliver greater reliability and performance for customers is a story about AWS realigning their network around the random graph theory.

P.S. I always feel excited about the networking stories, because I studied them in the university. Even though I haven’t worked closely with them since many years ago, and forgot almost everything about them.

#aws #networking

Unless you're super diligent with deprecation, you may be in a situation right now, when you need to migrate away from NGINX ingress.

Here's a great article that explains new Kubernetes API objects related to the GatewayAPI project that is here to replace Ingress.

Ingress API is not deprecated itself, but it won't be further developed either.

This article confuses the names for the community-led Ingress Nginx and the F5 NGINX ingress controller, but so do many of us: there are way too many nginx's in this world.

#kubernetes #networking #nginx

A new issue of the CatOps Digest is here!

https://newsletter.catops.dev/p/catops-digest-2026-05-29

#newsletter #digest

I Don’t Care if AI Wrote the Code. You Own It. is a reminder that you cannot call AI an idiot, if something goes wrong - you still bear the responsibility of what it does.

This short article just reiterates this statement, and points out that in this day an age, tests and validations are more important as ever before.

#ai #sre

Finding zombies in our systems: A real-world story of CPU bottlenecks is an interesting debugging story for those, who like technical detective tales.

P.S. I find Pinterest's technical blog quite interesting. It has many interesting articles out there.

#debug #aws #performance #kubernetes

​​For today’s Donations Monday, let’s finally close the fundraiser for two trucks that’s been going on for some time already.

​​​​https://send.monobank.ua/jar/3U1hBa5WPp

More info: https://www.instagram.com/p/DXpgaaWgH00

#donations #Ukraine

​​Today CatOps became 9 years old 🎉

You can send us a birthday present by donating to our current fundraiser!

https://send.monobank.ua/jar/3U1hBa5WPp

A nice article about chaos engineering that was shared in our chat.

The author uses some overly fluffy sentences, but the core of the article holds strong: in many cases, you don't need chaos engineering, and there are things that have better ROI, unless you have them already.

Personally, I'd also like to add that chaos engineering is not simply about breaking things - it's about experimentation. You don't just randomly switch off things, you build hypotheses and validate them. This is the boring, yet crucial part, that many oversee.

#chaos

​​What’s the difference between picking up litter after yourself and donating to the AFU's pickup trucks?

You're right — donating is easier, as you don't need to spend energy producing waste beforehand!

So, here's the link: ​​https://send.monobank.ua/jar/3U1hBa5WPp

More info: https://www.instagram.com/p/DXpgaaWgH00

#donations #Ukraine