krishnanayak Posted January 21, 2016 Share Posted January 21, 2016 A robots.txt document offers guidelines to web robots about the pages the site proprietor doesn't wish to be 'slithered'. Case in point, on the off chance that you didn't need your pictures to be recorded by Google and other web crawlers, you'd square them utilizing your robots.txt document. Quote Link to comment https://forums.phpfreaks.com/topic/300598-importance-of-robottxt/ Share on other sites More sharing options...
Muddy_Funster Posted January 21, 2016 Share Posted January 21, 2016 That's a statement (and not a very good one at that) - not a question. Was there a point to this? Quote Link to comment https://forums.phpfreaks.com/topic/300598-importance-of-robottxt/#findComment-1530180 Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.