jk11uk Posted September 17, 2008 Share Posted September 17, 2008 hi i made a google scraper a while ago, but google block it now. Sneakily they dont just block it, instead they send it dodgy data -nice. Anyway, i'd like to get around this so google cant tell its a scraper. So far i've: - changed the user agent so it cant detect the script that way - put in a random time delay so searching appears normal but it still doesn't work. I know they use cookies to track so my question is how do you have a script accept cookies and maybe after a while destroy them and accept new ones (or do what ever else is needed to look like a normal searcher!) ? Also any other ideas on how to make a scraper undetectable cheers! Link to comment https://forums.phpfreaks.com/topic/124655-google-cookie-tracking/ Share on other sites More sharing options...
jk11uk Posted September 18, 2008 Author Share Posted September 18, 2008 Can anyone help? Link to comment https://forums.phpfreaks.com/topic/124655-google-cookie-tracking/#findComment-644610 Share on other sites More sharing options...
JonnoTheDev Posted September 18, 2008 Share Posted September 18, 2008 It will be your IP address that is banned not anything else. Use a proxy but you'll only end up getting them IPs blocked aswell. Most of the free proxy addresses you can get hold of are banned by Google for this reason. Use the Google AJAX API on your own site in a manor that will not get you banned. Link to comment https://forums.phpfreaks.com/topic/124655-google-cookie-tracking/#findComment-644791 Share on other sites More sharing options...
jk11uk Posted September 19, 2008 Author Share Posted September 19, 2008 thanks for the tips i dont think it is the IP though, as then surely i wouldnt be able to use google search? also, i run another scraping program which doesn't get blocked (ranking checker). Link to comment https://forums.phpfreaks.com/topic/124655-google-cookie-tracking/#findComment-645479 Share on other sites More sharing options...
aschk Posted September 19, 2008 Share Posted September 19, 2008 Witht posting any of your code JK it's very difficult to tell exactly what you're doing and how we can assist to avoid your cookie (or other relevant) issue. Care to post some code to help us out? Link to comment https://forums.phpfreaks.com/topic/124655-google-cookie-tracking/#findComment-645529 Share on other sites More sharing options...
Recommended Posts
Archived
This topic is now archived and is closed to further replies.