Jump to content

Archived

This topic is now archived and is closed to further replies.

FrOzeN

Googlebot, and the like?

Recommended Posts

I've noticed on some forums, with online users it has a specific one for "GoogleBot".

I was thinking of implementing something like this, but maybe to also include some other main ones for yahoo and msn. I figured that this can be determined with $_SERVER["HTTP_AGENT"], but that can also be easily spoofed.

My other thought is that maybe they have a range of IP's which could be matched against on top of $_SERVER["HTTP_AGENT"] for a better/more assumed verification of it being real.

Any suggests for creating something like this? Or anyone know a little of IP's (or just ranges) which these bots use, and what bots they're from?

Share this post


Link to post
Share on other sites
There isn't a 100% reliable way. HTTP_USER_AGENT and HTTP_HOST are your best bets, but again, they are not 100% reliable.

Also note if you are intending on supplying different content to googlebot than you do for normal users, if found out you will receive automatic -1 google rank.

Share this post


Link to post
Share on other sites
What is the big deal about this? So what if somebody pretends to be Googlebot. I don't suppose that you grant Googlebot administrative privileges in your applications, so there wouldn't really be any idea in changing your HTTP_USER_AGENT to Googlebot.

Share this post


Link to post
Share on other sites
Doesn't google list the IPs their bots are usually on?

Can't remember where I saw the list.

Share this post


Link to post
Share on other sites
I don't know about lists from google itself, but I do know [url=http://iplists.com/]this guy[/url] keeps lists of SE crawler ip's for cloaking purposes.

[b]
DISCLAIMER:[/b] cloaking can get you banned from google.  ;)

Share this post


Link to post
Share on other sites

×

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.