For some odd reason my htaccess was banning some Bingbot and Yandexbot crawls, but there were others, such as University of Toronto, and I could not figure out why. I blamed my host provider, and I was incorrect. There was no other cache of IP blocks. It turned out to be, as usual, a user error. My user error.
Through the course of repeatedly bisecting my htaccess, I went through with my patient Big Weed, 12 times, until I isolated the errant code. It turned out to be syntactically correct but logically incorrect. As the syntax was fine, it passed my two htaccess syntax checkers. The error was, in CIDR format:
Or the difference of a missing “4” digit. What was this difference? The difference is huge. While the CIDR /24 blocks 256 IP addresses, the CIDR /2 range was interpreted as from 18.104.22.168 to 127.255.255.255, a wide swath of IP addresses, or, to be exact 1,073,741,824. That is 1 billion, 74 million IP addresses. On the plus side I am hoping that my site traffic will increase!
As the error was not syntactic but logical, it was difficult to detect and to fix. Let us hope that these errors are fewer in the future!