Tag: GoogleBot

Googlebot Errors caused by Errant htaccess rule

Difficult it was, this afternoon, after 7 days of Googlebot crawl error 500s, but I am learning. One htaccess regex line error was the cause. Hopefully it will go away.

I tried to compress some HTTP_USER_AGENT mod rewrite rules in my htaccess, into a single line, in order to shorten my htaccess, from:

RewriteCond %{HTTP_USER_AGENT} ^.*Go\ 1\.1 [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Go\!Zilla [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Go!Zilla [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Go\-Ahead\-Got\-It [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Go\-http [NC,OR]

to:
RewriteCond %{HTTP_USER_AGENT} ^.*Go[\ 1\.1|\!Zilla|\-Ahead\-Got\-It|\-http] [NC,OR]