What yacybot does
yacybot is the crawler used by YaCy, a peer-to-peer web search engine that is operated by its users rather than by a central company. Individual YaCy users run and maintain the crawler instances, which appear in server logs under the yacybot user-agent. The project states that yacybot follows the rules defined in a site's robots.txt file, and that after some hours all yacybots will obey new instructions.
How to control yacybot
Target yacybot in your robots.txt with the yacybot token. Rules apply per bot, so you can allow this one while blocking others, or the reverse.
Allow yacybot
User-agent: yacybot
Allow: /Block yacybot
User-agent: yacybot
Disallow: /Because YaCy crawler instances are run by many independent users, the project notes that a robots.txt change takes some hours to propagate before all yacybots obey it.
How to identify yacybot
Temso identifies yacybottraffic by matching the request's User-Agent header against this pattern. User-agent strings can be spoofed, so treat a match as an indication, not proof.
yacybotyacybot FAQ
What is yacybot?
yacybot is the crawler of YaCy, a peer-to-peer web search engine run by its users instead of a central operator. The crawler instances you see in your logs are maintained by individual YaCy users.
How do I block yacybot?
Add a robots.txt group with User-agent: yacybot and Disallow: /. YaCy states that its crawlers follow robots.txt, but since many independent instances exist, expect some hours to pass before all yacybots obey your new rules.
About Search Engine Crawlers
Classic search indexers that now also feed AI features: Google's AI Overviews and AI Mode build on Googlebot's index, and Microsoft Copilot builds on Bingbot's.
Blocking these removes you from both traditional rankings and the AI answers built on top of those indexes, usually the most costly block of all.
