Jump to content

importance of robot.txt?

Recommended Posts

A robots.txt document offers guidelines to web robots about the pages the site proprietor doesn't wish to be 'slithered'. Case in point, on the off chance that you didn't need your pictures to be recorded by Google and other web crawlers, you'd square them utilizing your robots.txt document.

Share this post

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now


Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.