CatOps

So, Amazon pushed back on the Financial Times report about AI causing outages. This is not news.

However, they now require a senior engineer's approval on the AI generated code pushed by non-senior engineers, apparently. It's not clear, who should review AI code generated by seniors, though.

- Financial Times (paywalled).
- Opinion on Xitter.
- Discussion on Hacker News.

This is an interesting twist on the whole AI adoption, and it would be very interesting to see where it would go from here. At least, at Amazon.

P.S. As a bonus, here's a book bundle about LLMs and some related stuff, so you could review AI-generated code more efficiently!

#ai #aws #culture

4 ways to use Argo CD and Terraform together is an article on how to pass data between Terraform (or OpenTofu) and ArgoCD (or any other GitOps tool for that matter). For example, if you're creating a new infrastructure component and need to pass its endpoints to an app.


They pointed out quite explicitly, that you shouldn't just pass raw secrets around. However, they didn't mention any secret storage for whatever reason. For example, you can use vault_generic_secret resource to store credentials in Vault and then something like the External Secrets Operator to fetch them from there. I am not saying, you should, but you can.

P.S. This article was shared in our chat. Come join us! The chat is in Ukrainian, and it's usually fun!

#terraform #argocd #cd #kubernetes

These days all the talks are about AI.

My (hypothetical) SRECon26 keynote is an article from Charity Majors with her advice on how SREs should approach AI.

BTW, I’ve heard good things about SRECon. I don’t know if the upcoming SRECon Americas is the same as SRECon in Dublin in October, but maybe I should check that one out!

#sre #ai

You know that I keep the most juicy articles for Friday, right?

AI Isn't Replacing SREs. It's Deskilling Them.

Here's the article. I leave you with that.


#sre #culture

A story from OpenAI on how they scale Postgres.

While this is an interesting read, and you can definitely borrow some ideas from there; I got an impression that OpenAI is moving towards Azure’s CosmosDB from Postgres.

If a new feature requires additional tables, they must be in alternative sharded systems such as Azure CosmosDB rather than PostgreSQL.

—-

While we’re happy with how far PostgreSQL has taken us, we continue to push its limits to ensure we have sufficient runway for future growth. We’ve already migrated the shardable write-heavy workloads to our sharded systems like CosmosDB. The remaining write-heavy workloads are more challenging to shard—we’re actively migrating those as well to further offload writes from the PostgreSQL primary.

Cosmos DB, from my understanding, is a document-oriented DB from Microsoft, similar to MongoDB.

#databases

An article from Alex Ewerlöf on how the Staff Engineer Archetypes can backfire.

Honestly, it’s not the best of his articles, but it’s a good illustration that you cannot “lock yourself” just on a single aspect of your work. Especially, once you’ve gotten high enough to levels such as a Staff Engineer.

#culture #career

Two bundles for you today:

- Python courses
- Cloud Practice Exams (mostly AI-related, tho)

#bundle

​​For today’s Donations Monday, I’d like to ask you to donate the downing of russian UAVs.

This is a fundraiser by Come Back Alive and they support different payment methods!

#donations #Ukraine

A new issue of the digest is here!

https://newsletter.catops.dev/p/catops-digest-2026-02-27

#newsletter #digest

Why etcd breaks at scale in Kubernetes is a great article that describes, what etcd is, how it works in nutshell, what are its limitations; and, most importantly, why does it all matter to Kubernetes.

I had an etcd cluster becoming read-only due to the backend quota. The fact that this happened on the 1st of January added insult to injury.

P.S. LearnKube is a great resource for all the Kubernetes related topics. I highly recommend reading their other articles as well, if you want to get more familiar with the technology!

#kubernetes #etcd

An article from OpenAI on how they created a complete project without any human-written code.

This is, of course, kind of marketing material for OpenAI, but it also has interesting points:

 code throughput increased, our bottleneck became human QA capacity.

 management is one of the biggest challenges in making agents effective at large and complex tasks. One of the earliest lessons we learned was simple: give Codex a map, not a 1,000-page instruction manual.

 the agent’s point of view, anything it can’t access in-context while running effectively doesn’t exist. Knowledge that lives in Google Docs, chat threads, or people’s heads are not accessible to the system. Repository-local, versioned artifacts (e.g., code, markdown, schemas, executable plans) are all it can see.

And the most important point, in my opinion:

 kind of architecture you usually postpone until you have hundreds of engineers. With coding agents, it’s an early prerequisite: the constraints are what allows speed without decay or architectural drift.
...
In a human-first workflow, these rules might feel pedantic or constraining. With agents, they become multipliers: once encoded, they apply everywhere at once.

In any case, it's an interesting read. Obviously, it's all related to a completely green field project. So, your mileage for decade-old monoliths may vary.

P.S. Also, according to the diagrams in this article, OpenAI uses VictoriaMetrics, which is also cool :)

#ai #programming

For today's donations Monday, I'd like to share once again the standing Monobank jar for FPV equipment.

This jar is for the unit in which a guy from my wife's hometown serves.

https://send.monobank.ua/jar/4WLw91UqFe

#donations #Monday

Apparently, AWS had at least two recent outages due to AI. It was originally reported by Financial Times, but their article is behind a paywall. If you’re subscribed, you can read it here.

Me seeing these news surprisingly coincided with me seeing this post on Reddit: Vibe coders passing responsibility on code reviewers.

And this is kinda true, scary, and reassuring at the same time.

True because it’s indeed very easy these days to generate a lot of code in almost any language.

Scary, because the meme about 5000+ lines PRs with LGTM stands true. While AI code reviewers can quite effectively catch typos and style issues, that humans kinda suck in catching; overly complex logic is usually Ok for them. Thus, we will face more outages in the nearest future, in my opinion.

Reassuring, because it means that those of you who “keep the lights on” are not going anywhere because of AI. In fact, quite the opposite.

#ai

Collaboration sucks is a nice Friday evening read about the ways we work together.

I think, this article has interesting thoughts, but as usual, you need to use your own judgement to understand the environment you’re in.

For example:

You’re the driver” is a key value for us at PostHog. We aim to hire people who are great at their jobs and get out of their way. No deadlines, minimal coordination, and no managers telling you what to do.
In return, we ask for extraordinarily high ownership and the ability to get a lot done by _yourself._ Marketers ship code, salespeople answer technical questions without backup, and product engineers work across the stack

This works great until end up with the codebase that has a unique flavor of the same wheels at every corner. Sure, there are ways of dealing with that, but you have to have those constraints beforehand.

However, the idea of limiting your collaboration and inviting only the relevant people into the decision making process, makes total sense. If your company growths, at some point it will grow beyond the point a single person can understand every aspect of your system. When it happens, sharing proposals to everyone wouldn’t, indeed, improve your collaboration and the team spirit; it would just generate noise. And when there is too much noise, it’s easy to lose important signals.

#culture

​​For those of you, who're into MySQL.

There's an open letter to Oracle to establish a foundation to take care of MySQL.

https://letter.3306-db.org/

You can subscribe to make your voice heard. I would say, it's an important thing to do, because, you know, community matters. Also, it doesn't require much work from your side at this point.

More information is available via that link above.

#databases #mysql

​​Support a friend of mine on the Frontline!

Last year, she chose the tough path: Combat Medic.
Now, she needs our help to secure critical medical supplies that can't wait for paperwork.

No donation is too small. Let’s help her save lives!

- Mono Jar: https://send.monobank.ua/jar/75jQXw6aYq
- Mono: 💳: 4874100025644306
- Privat: 💳: 5168745027810065

#donations #Ukraine

From CatOps with love: a new issue of our newsletter is here!

https://newsletter.catops.dev/p/catops-digest-2026-02-14

#digest #newsletter

Continuing the topic of Terraform, here's an article from a friend of mine on how to keep the versions of Terraform binaries, modules, and providers in check.


This article was already featured in Anton's newsletter, but it won't hurt to re-iterate. If you're into Terraform, make sure to subscribe, tho.

P.S. If you're hiring in DE, CH, LU, DK, or remote; he has his CV on his blog. You know what to do 😉

#terraform

Some results of a fun testing of different LLMs to generate Terraform code.

This article is old, but they have updated the results in mid 2025. Anyways, keep in mind that since then, LLMs evolved. So, even those results are not quite correct anymore.

Still, it’s an interesting test that you can also do yourself. Another point is that LLMs are already quite usable to generate Terraform code.

#terraform #ai

Two bundles on Humble Bundle:

- Linux for Seasoned Admins by O’Reilly. This is a reoccurring bundle, so check if you have it already before buying!
- Networking bundle by Pact. This is a video bundle to prepare for various certification exams.

#books #bundle