SLURP SPIDER

Discussion in 'Website Feedback' started by spiderfran286, Nov 27, 2008.

  1. spiderfran286

    spiderfran286 "Yes, squid pro roe..." VIP Junglist

    Joined:
    Nov 4, 2007
    Messages:
    5,676
    Likes Received:
    28
    Location:
    Sheffield
    WTF IS
    GOOGLE SLURP SPIDER
    AND YAHOO SPIDER
    ?!?
    ITS IN THE CURRENTLY ONLINE MEMBERS LIST
     
  2. Gumby

    Gumby New Member VIP Junglist

    Joined:
    Apr 7, 2006
    Messages:
    6,208
    Likes Received:
    10
    its just people viewing the forum from a google/yahoo search.



    excitement.
     
  3. perspective

    perspective Sausage Rolls VIP Junglist

    Joined:
    Aug 5, 2008
    Messages:
    1,154
    Likes Received:
    3
    Location:
    Portsmouth, UK
    in order for google to know what is on the web it needs to 'spider' it every now and then...

    It starts off with a load of links it already knows about, then goes to each one and caches the pages, then goes to all the links on that page and so on.

    So eventually it will see every page on the internet that has a link to it on another page on the internet.

    Explained quite well here: http://www.googleguide.com/google_works.html

    The forum software recognises the google spider ting from the HTTP headers it sends, and lets it view pages from the perspective of a registered user which is why it appears on the logged in users (y)
     
  4. Fes Rock

    Fes Rock Nothing.......... VIP Junglist

    Joined:
    Jul 21, 2005
    Messages:
    15,250
    Likes Received:
    140
    Location:
    U WOT M8?

    WOAH!
     
  5. $marty

    $marty Dexcell Tribal Leader VIP Junglist

    Joined:
    Dec 8, 2003
    Messages:
    18,827
    Likes Received:
    2,194
    Location:
    Watford
    sounds a bit like eagle eye to me.
     
  6. SIRUS

    SIRUS 変なひと VIP Junglist

    Joined:
    Feb 16, 2007
    Messages:
    5,321
    Likes Received:
    1,063
    that's how it cache's the pages.
     
  7. Fes Rock

    Fes Rock Nothing.......... VIP Junglist

    Joined:
    Jul 21, 2005
    Messages:
    15,250
    Likes Received:
    140
    Location:
    U WOT M8?
    cache rule everything around me!