gwolff2005 Posted August 3, 2009 Share Posted August 3, 2009 Hi I need your help. I have the following robots.txt in my root directory User-agent: * Disallow: /pics/ Disallow: /_mmServerScripts/ Disallow: /chat/ Disallow: /Connections/ Disallow: /test/ Disallow: /Tamara/ Disallow: /tpt/ Disallow: /forum/ Disallow: /guestbook/ Disallow: /news/ Disallow: /Newsletter/ Disallow: /notes/ Disallow: /meetguntmar.html Disallow: /events.html Disallow: /references.html Disallow: /media.html Disallow: /contact.html Disallow: /bookingcomany.html Disallow: /bookinghabitsofsuccess.html Disallow: /bookinglifemastery.html Disallow: /bookingstraightforward.html Disallow: /poll.html Disallow: /newsletter.html Disallow: /bookingevents.html Disallow: /feedback.html Disallow: /eventscalendar.html Disallow: /lifemastery.html Disallow: /habitsofsuccess.html Disallow: /straightforward.html Disallow: /indexmain.html Disallow: /seatingplan.html Disallow: /thankyou.html Disallow: /thankyoureg. Disallow: /indexmenu.html Disallow: /indexmain.php Disallow: /indexmenu.php But google still brings up indexmenu.htm. What can I do. What is wrong?? Thanks! Quote Link to comment https://forums.phpfreaks.com/topic/168622-solved-robotstxt-does-not-work/ Share on other sites More sharing options...
haku Posted August 3, 2009 Share Posted August 3, 2009 The first thing you should probably try is adding indexmenu.htm to the file. Quote Link to comment https://forums.phpfreaks.com/topic/168622-solved-robotstxt-does-not-work/#findComment-889496 Share on other sites More sharing options...
gwolff2005 Posted August 3, 2009 Author Share Posted August 3, 2009 sorry haku I meant indexmenu.html :-) My mistake! Google is still listing indexmenu.html Quote Link to comment https://forums.phpfreaks.com/topic/168622-solved-robotstxt-does-not-work/#findComment-889500 Share on other sites More sharing options...
infiniteacuity Posted August 11, 2009 Share Posted August 11, 2009 Have you re-submitted the site to google? Quote Link to comment https://forums.phpfreaks.com/topic/168622-solved-robotstxt-does-not-work/#findComment-895598 Share on other sites More sharing options...
BLaZuRE Posted August 12, 2009 Share Posted August 12, 2009 Just a guess, but is this valid?: Disallow: /thankyoureg. Maybe it stops there after not knowing what to do with the period? Quote Link to comment https://forums.phpfreaks.com/topic/168622-solved-robotstxt-does-not-work/#findComment-896091 Share on other sites More sharing options...
gwolff2005 Posted August 12, 2009 Author Share Posted August 12, 2009 Hi guys, thanks for your suggestions. I had to write no robots and noindex on top of every page and now they are not listed anymore. Google says that even you disallow the robot to go there it could be still indexed if another page links to them. Quote Link to comment https://forums.phpfreaks.com/topic/168622-solved-robotstxt-does-not-work/#findComment-896230 Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.