Report on today's disruptionWe experienced a service disruption between 19:40 and 20:01 UTC today that affected API transactions. The issue has been resolved, and all systems are back to normal. During this time, the core exchange and committer continued operating as expected, but the API server was unable to process transactions properly.
Before markets became volatile, we noticed that roughly one-third of all transactions were coming from what appeared to be automated bots engaged in wash trading. Even in low-volatility periods, this suspicious activity was driving unusually high TPS and contributing to instability. Later on, the markets went down quickly during the disruption.
We take full responsibility for the incident. All affected transactions have now been processed, and LLP orders were not impacted.
What we are doing:- We’re banning the identified bot accounts, revoking any associated points
- Reassessing our API volume quota and rate limit mechanisms to strengthen stability
- Putting in better retry logic in our API servers
- Making users whole with USDC compensations
If you were affected by this issue, please fill out the form provided. There’s no need to open a support ticket if it relates to this incident.
Form:
https://forms.gle/cJ1Wsiqzd8Ve3p5Z8