There’s a particular kind of infrastructure bug that’s deeply annoying to debug: the kind that only appears when your cluster is under load. Everything runs fine in steady state. Then you push a release, or traffic spikes and your autoscaler kicks in, and suddenly you’re chasing 502s that resolve on their own a few minutes later. Your first instinct is to blame the app. Then the deployment. Then the cluster. Then yourself.

We hit exactly this pattern at Simon AI. The culprit turned out to be something fundamental about how nginx handles DNS — and fixing it properly required waiting for a feature that was finally open-sourced at the end of 2024.

Here’s what we found, what we tried, and how we solved it.