Bentley4 Posted October 26, 2011 Share Posted October 26, 2011 Hi guys, Is it possible for a website to only be accessed by a link or typing the url? (in other words I don't want search engines to pick it up) How should I do this? B Quote Link to comment Share on other sites More sharing options...
requinix Posted October 26, 2011 Share Posted October 26, 2011 Any sufficiently motivated search engine bot is indistinguishable from a normal user. For the Lawful Good ones you can use a robots.txt in the root of your site with User-agent: * Disallow: / For the others you'll have to do user-agent checking and use IP address bans. But why don't you want search engines crawling the site? Quote Link to comment Share on other sites More sharing options...
Bentley4 Posted October 26, 2011 Author Share Posted October 26, 2011 I want to implement different versions of my site, one for potential employers, one for friends,... I would give them a link if I wanted them to check it out. When you said 'lawful good ones'; In other words it is possible for google, yahoo and bing? Quote Link to comment Share on other sites More sharing options...
requinix Posted October 26, 2011 Share Posted October 26, 2011 Yes, those three will. So you're saying you don't want potential employers to see what you're showing your friends? Quote Link to comment Share on other sites More sharing options...
Bentley4 Posted October 26, 2011 Author Share Posted October 26, 2011 Actually I'm not even going to use it for friends, just for people I don't know yet that I want to show my profile in a non-work related context. (different structure of website, work: more emphasis on skills, non-work: more emphasis on intrests and hobbies.) I also have a public one that everyone can look up, which gives less detail. Quote Link to comment Share on other sites More sharing options...
Philip Posted October 26, 2011 Share Posted October 26, 2011 If you're not linking it publicly, and neither are they, there is no way for the crawlers to find your pages without doing a lot of guessing. For example, http://www.example.com/123jhl342j3lk/preview/ is going to be pretty hard to guess Quote Link to comment Share on other sites More sharing options...
Bentley4 Posted October 26, 2011 Author Share Posted October 26, 2011 If that is the case I don't understand the following: I have a www.myname.info page. I have never linked to it anywhere and I'm pretty sure noone ever has because its relatively new and is blank. Yet when I type 'myname' it comes up as the 6th search result on google. I also have a flickr account with my name recently created, it contains no photos, haven't linked to it and I don't understand why anyone would link to it if it's still blank(so I assume noone did.) I can also find this account by typing in my name. Quote Link to comment Share on other sites More sharing options...
SparK_BR Posted October 27, 2011 Share Posted October 27, 2011 He means you must create a page like "hobbies.php" but in the main page you simply don't link to it so the bot will get to your index but if the index has no link to the other page, the bot won't see it Quote Link to comment Share on other sites More sharing options...
Bentley4 Posted October 27, 2011 Author Share Posted October 27, 2011 So basically you're saying that search engines(or rather webcrawlers associated to them) will always be able to find and list my main/homepage in search engines but not subdivisions if I don't link to them? Thnx Quote Link to comment Share on other sites More sharing options...
requinix Posted October 28, 2011 Share Posted October 28, 2011 Yes, but obscurity is not security. If the names are predictable like "admin" or "www2" or "private" then simply not linking to them doesn't guarantee (but greatly increases the odds) that they'll stay hidden. Also: somebody else might, for whatever reason, link to whatever "hidden" page on your site they want. Can't defeat that. Quote Link to comment Share on other sites More sharing options...
Philip Posted October 28, 2011 Share Posted October 28, 2011 Also: somebody else might, for whatever reason, link to whatever "hidden" page on your site they want. Can't defeat that. Sure, and that is very possible. However you could add something like an htaccess or simple php password to protect it. Quote Link to comment Share on other sites More sharing options...
requinix Posted October 28, 2011 Share Posted October 28, 2011 Sure, and that is very possible. However you could add something like an htaccess or simple php password to protect it. Which is basically what Bentley should be doing from the start. Hiding stuff doesn't really work. Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.