Skip to content

killer-robots

Kurz notiert: Yandex liest auch die killer-robots.txt:
[anon].yandex.com - - [15/Oct/2017:07:31:29 +0200] "GET /robots.txt HTTP/1.1" 200 145 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)"
[anon].yandex.com - - [15/Oct/2017:07:31:33 +0200] "GET /killer-robots.txt HTTP/1.1" 200 99 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)"