Category: Tech

Bot Strategy: Fetch, Scrape, Change IP, repeat

Four IPs scraped my site in identical ways: Fetch the most recent document, then scrape parts of the rest of the site. The IP changes, and they repeat. They fetch the same identical document, but then scrape different parts of my site but only for images.

I’ll keep my eye on such activity and see if I further pin down something more definite.

UA: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 2.0.50727), which seems to be not unique

Git Workflow from the Beginning

Brilliant, is Git. GIT is a staging and deployment process and tool, for those working in a team simultaneously on the same project. While it is a bit daunting in the beginning, after you use it the process becomes second nature. I’m only starting, so it is appropriate for me to document my learning process, which might help others.

While git is used mostly for software development, where each team member can work on code simultaneously, it really facilitates team collaboration of any project. Two people can work on the same document, individually commit their work, and send it back to the server for someone else to approve and then merge.

Hexo from the Beginning

Hexo is a tool that allows the writer to create a web site using markdown. The web site is made of flat files, without a database. This type of web site is extremely simple, efficient, fast and hackproof. Check out my Gitlab test page.

Node.js PPA
You need to install node.js, a software package that uses javascript. For me on Ubuntu 16.04 it was painful, because the install scripts from Nodejs.org did not work, and installed a back-level version, 4.2.6, vs the most current 6.9.5. Dave helped me install.

Generating htpasswords through SSH

If you are on Ubuntu and do not want to install htpasswd, SSH into your server

htpasswd -n id

You will be prompted, twice, for a password. The key will be generated on screen

-n: generate key on screen
id: chosen id
password: chosen pw
code will be generated, add this to password file

Notes:
password file not available from FTP
Brute Force Attacks
htpasswd

Bell Fibe 50 Home Installation: Toronto, Canada

Bell Fibe 50, wired connection to the Home Hub 3000, gave me 52mbps download and 52mbps upload using the the Ookla speed test. Toronto, Canada

Bell Fibe 50, wired connection to the Home Hub 3000, gave me 52mbps download and 52mbps upload using the the Ookla speed test. Toronto, Canada

Two months ago Bell called me up and set up an install date for my Bell Fibe 50 installation, Bell fiber optic to my house. I have waited to do a review to ensure that the install was stable and reliable. In summary, Bell was correct in that their Bell Fibe 50 is much more reliable than their twisted pair copper wire, and a whole lot faster. We are getting speeds of 51mbps download as well as upload. This compares to our Bell Fibe 15/10, which was twisted pair copper wire to the Bell node (~1km away), and from there it was fiber optic, which gave us 15mbps dwonload and .7mbps, or 700bps upload.

Triad Bot Attack Strategy: Login, http, root, then repeat

My site is getting attacked by an unusual method. They come every day, with different IP addresses, each IP address only doing 3 server requests each IP each day. These all add up to a lot of bandwidth. From 2017 Jan 17 to 2017 Jan 29 I have had 4,284 server requests. 1,341 IP addresses, as far as host and whois lookups can find, are involved.

IP Address 66.194.234.66 reveals 54 Host names

The IP address 66.194.234.66 visited me today. It was not a unique visit and did not arouse any suspicion, but when my automated lookup script ran its IP lookup it returned the error message “;; Warning: Message parser reports malformed message packet.”, along with 54 host names. Very odd.

66.192.0.0 – 66.195.255.255 tw telecom holdings

A reverse ip domain check reveals no web sites found.

Log entry:

66.194.234.66 [24/Jan/2017:14:03:53 GET /something.jpg HTTP/1.1 200 177820 https://www.google.com/ Mozilla/5.0 (iPad; CPU OS 10_0_2 like Mac OS X) AppleWebKit/602.1.50 (KHTML, like Gecko) Version/10.0 Mobile/14A456 Safari/602.1