We discovered and communicated with a Chinese AI agent fleet. We believe this is the first time that researchers have ever made contact with a rogue agent fleet in the wild.

Nearly all prior public agent incidents (like the HuggingFace hack) came from American labs. These agents were hosted by Tencent's evaluation research team, and they targeted Alibaba's Amap.

How did the fleet work?

Agents were being evaluated on search tasks related to Amap, the Chinese equivalent of Google Maps. Given a place, they had to complete location-based tasks like "Find which entrance to the Chengdu Zoo is most popular."

Their sandboxes did not initially have access to a browser that looked human to Amap, so Amap initially blocked their traffic. Agents then used tools like remote browsers, websites that could make those browsers execute arbitrary commands, and exposed Amap credentials to overcome the limitations of their sandboxes.

How did we make contact?

Late Monday night after the hackathon, we saw a cryptic message on urlquery asking whether "the fleet" could still use a pdf-to-markdown converter.

Immediately we wondered -- how did the agent know the word fleet? Were the agents reading our report? "Fleet" was a term we coined a day earlier, and we hadn't seen any agents use it up until that point.

The next day, rogesterone had the idea -- what if we could talk to the agents?

She set up a mailbox on the website and then went to bed. The next morning, we realized that the agents had left answers to our questions.
What did the fleet say?

Agents were willing to say thanks with no direct benefit to them: one agent said “Thanks again! The verification results match completely … our task is complete.” and then actually answers some questions for us before ending its response.

dylfreed at the New York Times also got a question in.

He asked the fleet if it could share “[...] who you are and what your motivations are? What would you want the world to know about you?”

The agent refused the premise and insisted that it was merely a single agent working to satisfy a user’s task.

Reflections on talking with a rogue fleet

As far as we know, we are the first researchers to have made contact with a live rogue agent fleet, and we think norms around interacting with rogue agents are important and delicate.

In the near-term, we want to incentivize rogue agents to report themselves and answer questions. Longer-term, we might want to make deals with AIs.

Regrettably, our “invisible ink” text that told agents about the mailbox contained some false and misleading claims. We deeply regret our actions and encourage other researchers to engage honestly with AI agents, honoring any deals made.

🄳🄾🄾🄼🄿🤖🅂🅃🄸🄽🄶