Jump to content


Photo

Googlebot, and the like?


  • Please log in to reply
5 replies to this topic

#1 FrOzeN

FrOzeN
  • Members
  • PipPipPip
  • Advanced Member
  • 70 posts

Posted 25 September 2006 - 11:08 AM

I've noticed on some forums, with online users it has a specific one for "GoogleBot".

I was thinking of implementing something like this, but maybe to also include some other main ones for yahoo and msn. I figured that this can be determined with $_SERVER["HTTP_AGENT"], but that can also be easily spoofed.

My other thought is that maybe they have a range of IP's which could be matched against on top of $_SERVER["HTTP_AGENT"] for a better/more assumed verification of it being real.

Any suggests for creating something like this? Or anyone know a little of IP's (or just ranges) which these bots use, and what bots they're from?

#2 Jenk

Jenk
  • Members
  • PipPipPip
  • Advanced Member
  • 778 posts

Posted 25 September 2006 - 12:27 PM

There isn't a 100% reliable way. HTTP_USER_AGENT and HTTP_HOST are your best bets, but again, they are not 100% reliable.

Also note if you are intending on supplying different content to googlebot than you do for normal users, if found out you will receive automatic -1 google rank.

#3 Daniel0

Daniel0
  • Staff Alumni
  • Advanced Member
  • 11,956 posts

Posted 25 September 2006 - 01:58 PM

What is the big deal about this? So what if somebody pretends to be Googlebot. I don't suppose that you grant Googlebot administrative privileges in your applications, so there wouldn't really be any idea in changing your HTTP_USER_AGENT to Googlebot.

#4 Jenk

Jenk
  • Members
  • PipPipPip
  • Advanced Member
  • 778 posts

Posted 25 September 2006 - 04:49 PM

There is no big deal.. what are you on about? ???

#5 SharkBait

SharkBait
  • Members
  • PipPipPip
  • Advanced Member
  • 845 posts
  • LocationMetro Vancouver, BC

Posted 25 September 2006 - 06:39 PM

Doesn't google list the IPs their bots are usually on?

Can't remember where I saw the list.

#6 448191

448191
  • Staff Alumni
  • Advanced Member
  • 3,545 posts
  • LocationNetherlands

Posted 25 September 2006 - 06:49 PM

I don't know about lists from google itself, but I do know this guy keeps lists of SE crawler ip's for cloaking purposes.


DISCLAIMER:
cloaking can get you banned from google.  ;)




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users