Jump to content

Robots.txt


liamloveslearning

Recommended Posts

Hi everyone, I have my robots.txt file in my root foler, my robots.txt is

 

User-agent: *
Allow: /
User-agent: *
Disallow: /eshot/
User-agent: *
Disallow: /newsletters/
User-agent: *
Disallow: /Connections/
User-agent: *
Disallow: /libraries/
User-agent: *
Disallow: /resources/
User-agent: *
Disallow: /sifr/
User-agent: *
Disallow: /SpryAssets/

 

in webmaster tools on google I receive the message "www.mywebiste.com Allowed by line 2: Allow: /

Detected as a directory; specific files may have different restrictions"

 

Does this mean where I have my allow all, that its contradicting my disallow restrictions?

Link to comment
Share on other sites

I took the freedom to edit your robots.txt file,

 

try this :

User-agent: *
Disallow: /eshot/
Disallow: /newsletters/
Disallow: /Connections/
Disallow: /libraries/
Disallow: /resources/
Disallow: /sifr/
Disallow: /SpryAssets/

 

That's basically is going to do all you're looking for, it will disallow those directories in all search engines,

Link to comment
Share on other sites

This thread is more than a year old. Please don't revive it unless you have something important to add.

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.