Block Yandex.ru bot crawling your website

Recently I’ve had a client who has had problems with the Russian search engine Yandex crawling their website and taking up server resources and bandwidth. Yandex.ru is a popular search engine in Russia unfortuently for many Yandex ignores the Robot.txt file therefore using the “official” method is not an option.

If you have a busy forum or website with hundreds of pages you’ll find that Yandex.ru is taking up more and more site resources by indexing up to 90 pages every 15 minutes.

By following my instructions you can stop Yandex.ru indexing your website. This needs to be placed in the .htaccess file.

SetEnvIfNoCase User-Agent “^Yandex*” bad_bot

Order Deny,Allow

Deny from env=bad_bot

Using this method saves you the trouble of having to find and block Yandex.ru IP addresses and block each individually which would only work for a limited time anyway.

If you need help implementing this find out my Web Development fees today.

Brian Martin