What's the AI Bot toll on Emaculation?

[adespoton]

I just finished reading https://arstechnica.com/ai/2025/03/devs ... countries/

It made me realize something: go on Microsoft Copilot and ask it a question regarding Mac emulation, and guess where its data comes from?

You can ask it about the usernames of anyone on this forum, and it will give you an up-to-date summary of the contributions that username has made to this website.

And then I realized: in order for Copilot to have this level of insight, it needs to be crawling the website constantly. And it's only one AI service.

So, mostly a question for Clockwise, but is that load having an impact here? Is there anything we can do to mitigate it?

[MiaM]

I don't know any actual fact about this topic, but I would think that any AI that needs crawling and who is affiliated with any search engine (Microsoft - Bing, Google - Google (obviously) and so on) just uses the same crawl as the search engine.

[Ronald P. Regensburg]

Search bots have, and always had, access to everything that can be seen by guest users, Some bots, like Google and Bing, visit us several times a day. A regular Google search on mac emulation will usually show this site high in the results. With AI around, search results will inevitably become more elaborate.

Some time ago Clockwise did block a couple of suspicious bots. Maybe he can check again which bots visit us.

[adespoton]

The troublesome ones that other sites have been seeing are ones that try to look like humans, so they're a lot harder to spot from their source IPs or user agents, but fairly obvious from their browsing patterns, as they repeatedly go through the entire site tree. I'm guessing that for whatever reason, those bots haven't been targeting here yet, despite Google and MS considering Emaculation.com the authoritative site for everything to do with Mac emulation.

[Ronald P. Regensburg]

We did have a few first posts of new users that seemed on topic at first sight, but that could have been written by AI. The messages "felt" strange and contained bits and pieces of earlier posts, put together in a new message.

You have not seen these because the first three posts of a new user are quarantined until accepted by one of the moderators. If the message is not accepted, usually because of spamming, the user is deleted.

[ClockWise]

Last year (and the year before) I blocked a ton of bots that were scraping the site 24/7. They were particularly bad since they didn't wait very long between calling pages. That caused slowdowns and worse. A few of them were from dodgy AI firms. Many more were from dodgy SEO firms.

Since there haven't been any major slowdowns I haven't tried blocking anything lately.

To be honest, I think OpenAI and Microsoft and Musk and the other big players have scraped the entire internet many times. They have everything.