Today was web geeking day for me. I had a few things on my todo list that I've wanted to take care of forever. The first was to start rebuilding rules for blocking bad web spiders (as I posted about earlier). Mark Pilgrim posted a great article on blocking spambots and spotbots a few months ago that is a great starting point. So first I started building up my rules. I'm mostly watching my logs to look for things that don't look like they are behaving well and adding rules based on that along with using the article as a guide for things to look for. So far so good.
The next thing I wanted to do was block people from hotlinking to images on my machine. More and more people have been grabbing images this way and it's getting on my nerves. I could care less if they copied them down to their own server and used them. I just don't want them using me as a hosting service. After looking around for a bit I found this article on using mod_rewrite to block hot linked images. For now people who do this will get a very nice image. After I get bored with that it'll just give them a 404 or something like that. Maybe I'll redirect them to where they came from just to confuse things even more.
And finally, I set up a redirect rule to point from some of my old archives to the current ones. It isn't perfect and doesn't catch everything, but does catch some things. It also tells search engines that the page has moved, so my old pages should vanish eventually. One thing I'm curious about now though is the 301 status for pages. 301 means that the page has moved to a new location and provides that location. Search engines should respect it when crawling (and many do I believe). One thing I think would be cool is if browsers would notice that a bookmark has changed that way. So you follow a bookmark to a page and it gives a 301, it should prompt you to change the bookmark or something. I also think RSS aggregators should do something like this too. It would be nice not to have to worry about keeping old redirect rules around forever.
