Block Bad Bots Apache: cPanel Configuration Guide

Bad bots? Recently, we’ve seen more and more traffic from bad bots. Depending on who you ask, as much as 50% of all internet traffic is caused by bots — both malicious and good. Unfortunately, all of this bot activity can easily overload a cPanel server, especially if multiple sites are hit at once.

You can reduce the impact of bad bots by implementing effective bot-blocking techniques. This guide shows you how to block bad bots in Apache through cPanel’s interface, with options for both server-wide configuration and per-site .htaccess rules.

Server-Wide Method: Apache Pre-Virtual Host Configuration

The simple method we use is to add a block rule to the pre-virtual host configuration in Apache. This blocks bad bots across all sites on your cPanel server.

Adjust this to block the bots that are causing you issues. After adding, just apply and restart Apache.

<IfModule mod_headers.c>
<Location />
SetEnvIfNoCase User-Agent "Baiduspider" bad_bot
SetEnvIfNoCase User-Agent "YandexBot" bad_bot
SetEnvIfNoCase User-Agent "MJ12bot" bad_bot
SetEnvIfNoCase User-Agent "AhrefsBot" bad_bot
SetEnvIfNoCase User-Agent "SemrushBot" bad_bot
SetEnvIfNoCase User-Agent "BLEXBot" bad_bot
SetEnvIfNoCase User-Agent "Sogou Spider" bad_bot
SetEnvIfNoCase User-Agent "SeznamBot" bad_bot
<RequireAll>
Require all granted
Require not env bad_bot
</RequireAll>
</Location>
</IfModule>

Per-Site Method: Using .htaccess to Block Bad Bots in cPanel

If you need to block bad bots for individual sites rather than server-wide, you can use .htaccess rules. This approach works when you don’t have root access or want different blocking rules for different domains.

Add this to your site’s .htaccess file (typically in public_html):

RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} (Baiduspider|YandexBot|MJ12bot|AhrefsBot|SemrushBot|BLEXBot|Sogou\ Spider|SeznamBot) [NC]
RewriteRule .* - [F,L]

List of Bad Bots to Block

The bots in our examples are commonly blocked because they consume significant resources without providing value to most sites. Here’s what each does:

• Baiduspider — Chinese search engine crawler, often unnecessary for non-Chinese sites
• YandexBot — Russian search engine, aggressive crawling patterns
• MJ12bot — Majestic SEO crawler, heavy resource usage
• AhrefsBot — SEO tool crawler, very aggressive
• SemrushBot — Marketing tool crawler, frequent requests
• BLEXBot — Link analysis bot, questionable value
• Sogou Spider — Chinese search crawler
• SeznamBot — Czech search engine

Monitor your server logs to identify which bots are hitting your sites hardest. Block only what’s causing problems — some bots may be valuable depending on your audience.