London Fixed-gear and Single-speed


 
Go Back   London Fixed-gear and Single-speed > Members List

Joe.S 4th Category

Joe.S

Visitor Messages

Showing Visitor Messages 1 to 1 of 1
  1. VelocityBoy
    12th April 2008 - permalink
    VelocityBoy
    Yahoo Slurp Spider.

    Right quick explanation:
    When you search for things on Yahoo (or any other search engine, but we'll go with Yahoo for the explanation), Yahoo doesn't trawl the entire then and there to find what you want. What Yahoo does is crawl in advance and grab a copy of every page on the internet. It can then index those pages, so that when you search Yahoo checks their index to see whether a page exists that matches your search terms.

    So what you are seeing is the Yahoo script sending out requests to ask for every page we have one by one. All search engines do this.

    However, other search engines might use a single machine to do it so many requests only
    show up as one guest user. Yahoo uses any machine in their system that has a spare moment. So many Yahoo machines send requests here and appear as many guest users.

    Google actually send more requests, but as they only show up once no-one realises. Yahoo ask for about 4,000 pages each day.

Statistics

Total Posts
Visitor Messages
General Information
  • Last Activity: 17th July 2008
  • Join Date: 29th November 2007
  • Referrals: 0

Friends

Showing Friends 1 to 1 of 1
Creative Commons License All times are GMT. The time now is 11:20.