vickree Posted April 18, 2008 Share Posted April 18, 2008 Can .php pages be found in google We need to make our site not available in search, is there a code for this Please, let me know ASAP!!! Link to comment https://forums.phpfreaks.com/topic/101742-privacy/ Share on other sites More sharing options...
Daniel0 Posted April 18, 2008 Share Posted April 18, 2008 Yes they can, but you can tell the spiders to not index your pages using a robots.txt file. User-agent: * Disallow: / See: http://www.robotstxt.org/ Link to comment https://forums.phpfreaks.com/topic/101742-privacy/#findComment-520529 Share on other sites More sharing options...
vickree Posted April 18, 2008 Author Share Posted April 18, 2008 Yes they can, but you can tell the spiders to not index your pages using a robots.txt file. User-agent: * Disallow: / See: http://www.robotstxt.org/ Thanks so much. Do you know if there is a way to test this? Link to comment https://forums.phpfreaks.com/topic/101742-privacy/#findComment-520613 Share on other sites More sharing options...
Daniel0 Posted April 18, 2008 Share Posted April 18, 2008 You cannot really test it. It's merely a way of telling them to not crawl your site. They don't have to abide, but the vast majority does. Link to comment https://forums.phpfreaks.com/topic/101742-privacy/#findComment-520637 Share on other sites More sharing options...
tinker Posted April 18, 2008 Share Posted April 18, 2008 check your server logs for agent and referer Link to comment https://forums.phpfreaks.com/topic/101742-privacy/#findComment-520645 Share on other sites More sharing options...
Recommended Posts
Archived
This topic is now archived and is closed to further replies.