Like most people, I’ve had a Yahoo! ID and mail account for a long time. In addition to being useful for throw-away things like registering for software downloads, I use it for all of the Yahoo! Groups that I’ve subscribed to.
I recently learned how to use wget for retrieving single web pages or entire mirrors and thought I’d capture some of my thoughts and notes.
While reviewing some of my referrer logs recently, I noticed a couple of hits to specific picture pages based on Google search results. I decided it was time to learn how to keep search engines away from certain parts of my site. Following are some things I learned along the way.