FrOzeN Posted September 25, 2006 Share Posted September 25, 2006 I've noticed on some forums, with online users it has a specific one for "GoogleBot".I was thinking of implementing something like this, but maybe to also include some other main ones for yahoo and msn. I figured that this can be determined with $_SERVER["HTTP_AGENT"], but that can also be easily spoofed.My other thought is that maybe they have a range of IP's which could be matched against on top of $_SERVER["HTTP_AGENT"] for a better/more assumed verification of it being real.Any suggests for creating something like this? Or anyone know a little of IP's (or just ranges) which these bots use, and what bots they're from? Link to comment https://forums.phpfreaks.com/topic/21961-googlebot-and-the-like/ Share on other sites More sharing options...
Jenk Posted September 25, 2006 Share Posted September 25, 2006 There isn't a 100% reliable way. HTTP_USER_AGENT and HTTP_HOST are your best bets, but again, they are not 100% reliable.Also note if you are intending on supplying different content to googlebot than you do for normal users, if found out you will receive automatic -1 google rank. Link to comment https://forums.phpfreaks.com/topic/21961-googlebot-and-the-like/#findComment-98104 Share on other sites More sharing options...
Daniel0 Posted September 25, 2006 Share Posted September 25, 2006 What is the big deal about this? So what if somebody pretends to be Googlebot. I don't suppose that you grant Googlebot administrative privileges in your applications, so there wouldn't really be any idea in changing your HTTP_USER_AGENT to Googlebot. Link to comment https://forums.phpfreaks.com/topic/21961-googlebot-and-the-like/#findComment-98161 Share on other sites More sharing options...
Jenk Posted September 25, 2006 Share Posted September 25, 2006 There is no big deal.. what are you on about? ??? Link to comment https://forums.phpfreaks.com/topic/21961-googlebot-and-the-like/#findComment-98333 Share on other sites More sharing options...
SharkBait Posted September 25, 2006 Share Posted September 25, 2006 Doesn't google list the IPs their bots are usually on?Can't remember where I saw the list. Link to comment https://forums.phpfreaks.com/topic/21961-googlebot-and-the-like/#findComment-98423 Share on other sites More sharing options...
448191 Posted September 25, 2006 Share Posted September 25, 2006 I don't know about lists from google itself, but I do know [url=http://iplists.com/]this guy[/url] keeps lists of SE crawler ip's for cloaking purposes.[b]DISCLAIMER:[/b] cloaking can get you banned from google. ;) Link to comment https://forums.phpfreaks.com/topic/21961-googlebot-and-the-like/#findComment-98432 Share on other sites More sharing options...
Recommended Posts
Archived
This topic is now archived and is closed to further replies.