I discovered that 1000s of bots are actively monitoring X for new links posted, probably not just on my account

I noticed when everytime I'd share one of my sites new features or pages, the page would get a bit slower just after posting, then I asked Claude to analyze it

I tweeted a test URL as a reply somewhere deeply hidden in a thread (so not even a main post)

And within 8 seconds there's thousands of crawlers opening the page!

One is a military intelligence firm, that scans social media for breaking news and sells alerts to newsrooms, banks, companies and government, it specifically follows accounts with many followers

Another one is a social media monitoring company that tracks what people say online

I knew of course high frequency trading firms use online news to trade fast on things, but interesting to see it in real life

Twitterbot (X) of course is first, within 0 seconds of sharing, it opens it (probably when I wrote the tweet)

The first non-X bot arrives just 8.40s later, which I think is pretty fast considering it's a reply by me deeply hidden in a thread somewhere

TL;DR they're watching you!

If it's a main tweet (so not a reply), it's even faster

Within 0.4 seconds there is non-X crawlers opening the link!

𝕏 https://x.com/levelsio/status/2108196980983795995

Read on levels.io →