They have not caught on yet, as many Chinese search engines offer no explanation in English of their services, yet their bots are out there crawling all sites on the internet. I am trying to kill as many bots, scrapers, spiders, crawlers, and checkers as possible, and if I don’t find an explanation of what a site does, I’ll ban the user agent from my site. This might seem a little unfair, but I don’t want bots eating my bandwidth, and to have my ISP force me to pay more for hosting, all because they wish to crawl my site.
