Tag: Atlantic.net

Bot Strategy: Fetch, Scrape, Change IP, repeat

Four IPs scraped my site in identical ways: Fetch the most recent document, then scrape parts of the rest of the site. The IP changes, and they repeat. They fetch the same identical document, but then scrape different parts of my site but only for images.

I’ll keep my eye on such activity and see if I further pin down something more definite.

UA: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 2.0.50727), which seems to be not unique