finding bots via useragent SQL

[Crillster]

Yo!

 

Welcome to my first post! Hope its in the right place  :-\

 

Problem: I am creating a stats counter for my website and im having abit of trouble with bots! I've decided that since bots can't read JS to do a little AJAX from the index page to a statcounter.php file sending over all the data required. I've found that the number of hits that im getting from this new stat counter and reduced dramatically! I've done a check to find a ratio of bots to people using the following code -

 

$rs=mysql_query("SELECT useragent FROM logline WHERE useragent LIKE '%bot%'");

 

echo mysql_num_rows($rs)."<br>"; //outputs 1724

 

$rs=mysql_query("SELECT useragent FROM logline");

 

echo mysql_num_rows($rs);  // outputs 22997

 

Now dont worry about the statcount.php and the AJAX by the time i've explained it all i would have fixed it. The question is:

 

Is the sql query "SELECT useragent FROM logline WHERE useragent LIKE '%bot%'" a valid way for searching for bots via useragent? If not how can I refine it?

 

I really need to know if the ratio that the above code is telling me is correct and if i should be looking for a problem in the statcounter.php file, or if my website really is as shit as the new stats are telling me lol.

 

Cheers for any help in advance 

[trq]

Bots do not use any user agent simply called bot. There are a variety of user agents they do however use. If you Google common bot user agent or similar I'm sure you'll find a list.

[Crillster]

Cheers thorpe,

 

Just to clear something up.. i understand that "Bots do not use any user agent simply called bot" but i did assume that the string 'bot' would appear in the user agent.

)

I'll have a google for some bot user agents and see where that gets me. Thanks for the info

 

Any more for any more?

[trq]

[but i did assume that the string 'bot' would appear in the user agent.

 

You know what happens when you assume.

[Crillster]

ok.. for people that are interested this is the new shit!

 

Having done some googling i found that the user agent will often have the following strings in them:

 

mysql_select_db("snail_stats", $conn);

 

$rs=mysql_query("SELECT useragent FROM logline

WHERE useragent LIKE '%bot%'

|| useragent LIKE '%Crawler%'

|| useragent LIKE '%Spider%'

|| useragent LIKE '%Ask Jeeves%'

|| useragent LIKE '%Search%'

|| useragent LIKE '%Indexer%'

|| useragent LIKE '%Archiver%'");

 

 

echo mysql_num_rows($rs)."<br>"; //outputs 15566

 

$rs=mysql_query("SELECT useragent FROM logline");

 

echo mysql_num_rows($rs); // outputs 22997

 

This is much more like it!

 

Thanks for the help thorpe - u da man!