A load test on monday.com's AI Reports feature revealed a critical race condition caused by a hidden singleton pattern. When multiple users simultaneously generated reports, a WorkdocsAPIService registered as a singleton shared mutable state across concurrent requests, causing workdoc IDs to override each other and trigger 400 errors. The bug remained undetected in production due to low adoption rates and high pod count, which minimized collision probability. The investigation traced through multiple hypotheses before discovering the singleton registration issue, highlighting the importance of load testing, end-to-end concurrent testing, and preferring stateless class designs in asynchronous environments.
Post #1955
14.5K

Unmasking a hidden singleton
- ❤ 14
- 👍 6