Spiders, spam, and abuse: be gone!
I have reached the boiling point. I will tolerate spam and spiders no longer. Period.
Any and all systems that send spam to our systems, users, or networks or try to brute-force accounts on systems with domains we host (such as this), will be permanently banned at the firewall on both port 25 and port 80. In the last 4 days, I have banned 704 separate IPs and netblocks. The more we block and ban, the faster they come at us, and so we continue to block more.
I report every single one to the ‘abuse@provider.tld’ address automatically, and if the provider doesn’t support the ‘abuse’ address, I block them, permanently. I’ve reported thousands of spam messages to the ‘abuse’ address at major providers, such as cox.net, verizon.net, aol.com, hotmail.com, and others. These providers are doing nothing about the spam coming through their systems.
I’m also getting tired of these incessant web spiders, crawlers and robots hitting the webservers, ignoring robots.txt and without properly asking if they can spider the sites we host, first. Spiders without a proper identification in the UserAgent string (including a URL to the info page, or a functioning contact email address), or without asking us if they can spider our content first, are also being blocked on port 80.
These spiders are continuously spidering the same content, including VERY large downloads, over and over and over, daily. There can be absolutely no reason for this, and hence, banned.
I’m rapidly tiring of this abuse. 3,000+ spam messages a day is just out of control.