I've checked out Project Honeypot and am going to be implementing it on my site. It's basically a cloud honeypot for spam/bad bots. You run it on your site. In exchange you get access to their bad bot/spam traffic blacklist and can automatically block bad traffic. Here's how it works...
1) Website(s) install project honeypot on their website after creating an account
2) Project honeypot sets up some traps for bots and spam traffic. Good example - hidden link on homepage pointing to /admin/. Robots.txt says NOBODY goes to /admin/. Bot is super curious and goes to admin.
3) Bot IP is collected and banned (firewall rules I believe). Not only that, but the IP address of the bad visitor is sent to the cloud blacklist along with what they did.
4) IF that bot tries to visit my site (running honeypot) I can either provide an extra level of scrutiny or straight up block them
It seems fairly well thought out. It deploys decoy pages/emails and syncs information back to the project. It blocks harvesters, spammers, dictionary attackers, and comment spammers (sorry BHW). It's very well thought out... for example on the email trap page it does the following...
1) Each visitor gets a unique email generated on the page. Example:
[email protected]. Each visitor IP is logged to that email address. If an email is ever directed to
[email protected] - that IP is blacklisted. They harvested your email address and tried to send to it.
2) Since your honeypot is syncing with the cloud, that IP is not blacklisted on every site signed up with project honeypot.
Effective IMHO. That failing here's what I'd use if you can code
1) Robots enforcement - Specifically list juicy directores (eg /admin/) in the robots.txt and forbid them to all robots. Have any visitor to /admin/ automatically banned (firewall/IPTables) for say 1 day.
2) Hidden links - make links people can't see. Only bots will pick them up. Same solution as above. This would also be effective with mass downloaders of your website.
I wouldn't extend the ban for very long (i.e. lifetime) for a couple reasons. First is to keep the banned list of IP's small. Second is to ensure you still get legit visitors. IP's change all the time. Be flexible.
I'm sure there are other ideas you can develop. This should get you started.