SLURP SPIDER

Discussion in 'Website Feedback' started by spiderfran286, Nov 27, 2008.

  1. spiderfran286

    spiderfran286 "Yes, squid pro roe..."

    Joined:
    Nov 4, 2007
    Messages:
    5,678
    Likes Received:
    28
    Location:
    Sheffield
    WTF IS
    GOOGLE SLURP SPIDER
    AND YAHOO SPIDER
    ?!?
    ITS IN THE CURRENTLY ONLINE MEMBERS LIST
     
  2. Gumby

    Gumby New Member

    Joined:
    Apr 7, 2006
    Messages:
    6,208
    Likes Received:
    10
    its just people viewing the forum from a google/yahoo search.



    excitement.
     
  3. perspective

    perspective Sex Drugs & Sausage Rolls

    Joined:
    Aug 5, 2008
    Messages:
    1,154
    Likes Received:
    3
    Location:
    Portsmouth, UK
    in order for google to know what is on the web it needs to 'spider' it every now and then...

    It starts off with a load of links it already knows about, then goes to each one and caches the pages, then goes to all the links on that page and so on.

    So eventually it will see every page on the internet that has a link to it on another page on the internet.

    Explained quite well here: http://www.googleguide.com/google_works.html

    The forum software recognises the google spider ting from the HTTP headers it sends, and lets it view pages from the perspective of a registered user which is why it appears on the logged in users (y)
     
  4. Fes Rock

    Fes Rock Nothing..........

    Joined:
    Jul 21, 2005
    Messages:
    15,258
    Likes Received:
    138
    Location:
    U WOT M8?

    WOAH!
     
  5. $marty

    $marty Dexcell Staff Member

    Joined:
    Dec 8, 2003
    Messages:
    18,697
    Likes Received:
    1,944
    Location:
    Watford
    sounds a bit like eagle eye to me.
     
  6. SIRUS

    SIRUS 変なひと

    Joined:
    Feb 16, 2007
    Messages:
    5,157
    Likes Received:
    873
    that's how it cache's the pages.
     
  7. Fes Rock

    Fes Rock Nothing..........

    Joined:
    Jul 21, 2005
    Messages:
    15,258
    Likes Received:
    138
    Location:
    U WOT M8?
    cache rule everything around me!