mysterbx Posted April 8, 2009 Share Posted April 8, 2009 Hy, Is there a way to somehow block any (exept my own) site or program from crawling my site? Something with htaccess or php, i dont know, protect my data somehow, from curl/get_file_contents/include and so one... Link to comment https://forums.phpfreaks.com/topic/153232-block-any-remote-access-method/ Share on other sites More sharing options...
schilly Posted April 8, 2009 Share Posted April 8, 2009 you can use a robots.txt to stop bots from indexing your site. i'm not sure on the specifics. just google it. Link to comment https://forums.phpfreaks.com/topic/153232-block-any-remote-access-method/#findComment-805019 Share on other sites More sharing options...
mysterbx Posted April 8, 2009 Author Share Posted April 8, 2009 "So don't try to use /robots.txt to hide information. ", said robots.txt but thats exactly what i need... Link to comment https://forums.phpfreaks.com/topic/153232-block-any-remote-access-method/#findComment-805037 Share on other sites More sharing options...
schilly Posted April 8, 2009 Share Posted April 8, 2009 create a script to check the user agent then decide whether to load or not load content? Link to comment https://forums.phpfreaks.com/topic/153232-block-any-remote-access-method/#findComment-805043 Share on other sites More sharing options...
redarrow Posted April 8, 2009 Share Posted April 8, 2009 <html> <head> <meta name="robots" content="noindex" /> <title>Don't index this page</title> </head> Link to comment https://forums.phpfreaks.com/topic/153232-block-any-remote-access-method/#findComment-805044 Share on other sites More sharing options...
mysterbx Posted April 9, 2009 Author Share Posted April 9, 2009 create a script to check the user agent then decide whether to load or not load content? you can change your user agent or set a fake one with curl... Link to comment https://forums.phpfreaks.com/topic/153232-block-any-remote-access-method/#findComment-805610 Share on other sites More sharing options...
PFMaBiSmAd Posted April 9, 2009 Share Posted April 9, 2009 What exact problem are you having? The whole point of the Internet is to publish content. For your content to be visible on your pages, it must be accessible to the browser to request to render on the page. Even if you use a member/login system, someone could create an account and then use CURL to log in and access the content. Link to comment https://forums.phpfreaks.com/topic/153232-block-any-remote-access-method/#findComment-805621 Share on other sites More sharing options...
mysterbx Posted April 9, 2009 Author Share Posted April 9, 2009 there is no problemo, i just need protection i just need to block anything that is not human, i tried to access www.katz.cd with curl, tried all kinds of methods/options, just couldnt get any reply from it, thats what i need... Its something with htaccess i gues... Link to comment https://forums.phpfreaks.com/topic/153232-block-any-remote-access-method/#findComment-805928 Share on other sites More sharing options...
Recommended Posts
Archived
This topic is now archived and is closed to further replies.