SLURP SPIDER

spiderfran286

"Yes, squid pro roe..."
VIP Junglist
Messages
5,643
WTF IS
GOOGLE SLURP SPIDER
AND YAHOO SPIDER
?!?
ITS IN THE CURRENTLY ONLINE MEMBERS LIST
 

Gumby

New Member
VIP Junglist
Messages
5,671
its just people viewing the forum from a google/yahoo search.



excitement.
 

perspective

Sausage Rolls
VIP Junglist
Messages
1,151
in order for google to know what is on the web it needs to 'spider' it every now and then...

It starts off with a load of links it already knows about, then goes to each one and caches the pages, then goes to all the links on that page and so on.

So eventually it will see every page on the internet that has a link to it on another page on the internet.

Explained quite well here: http://www.googleguide.com/google_works.html

The forum software recognises the google spider ting from the HTTP headers it sends, and lets it view pages from the perspective of a registered user which is why it appears on the logged in users (y)
 

Fes Rock

Nothing..........
VIP Junglist
Messages
15,091
in order for google to know what is on the web it needs to 'spider' it every now and then...

It starts off with a load of links it already knows about, then goes to each one and caches the pages, then goes to all the links on that page and so on.

So eventually it will see every page on the internet that has a link to it on another page on the internet.

Explained quite well here: http://www.googleguide.com/google_works.html

The forum software recognises the google spider ting from the HTTP headers it sends, and lets it view pages from the perspective of a registered user which is why it appears on the logged in users (y)

WOAH!
 
Top