From 5c41eaf7e9aaf2cd540af6f713f1630438292797 Mon Sep 17 00:00:00 2001 From: bochard Date: Wed, 17 Dec 2025 10:47:03 +0800 Subject: [PATCH] updated robots.txt that blocks internet archive --- robots.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/robots.txt b/robots.txt index 68f41d3..853b8fe 100644 --- a/robots.txt +++ b/robots.txt @@ -1199,7 +1199,7 @@ Disallow:/ User-agent: anthropic-ai Disallow:/ User-agent: archive.org_bot -Disallow:/ +Disallow: User-agent: arquivo-web-crawler Disallow:/ User-agent: arquivo.pt -- 2.39.5