Jump to content

Search the Community

Showing results for tags 'robots.txt'.

  • Search By Tags

    Type tags separated by commas.

  • Search By Author

Content Type


Forums

  • Welcome to PHP Freaks
    • Announcements
    • Introductions
  • PHP Coding
    • PHP Coding Help
    • Regex Help
    • Third Party Scripts
    • FAQ/Code Snippet Repository
  • SQL / Database
    • MySQL Help
    • PostgreSQL
    • Microsoft SQL - MSSQL
    • Other RDBMS and SQL dialects
  • Client Side
    • HTML Help
    • CSS Help
    • Javascript Help
    • Other
  • Applications and Frameworks
    • Applications
    • Frameworks
    • Other Libraries
  • Web Server Administration
    • PHP Installation and Configuration
    • Linux
    • Apache HTTP Server
    • Microsoft IIS
    • Other Web Server Software
  • Other
    • Application Design
    • Other Programming Languages
    • Editor Help (PhpStorm, VS Code, etc)
    • Website Critique
    • Beta Test Your Stuff!
  • Freelance, Contracts, Employment, etc.
    • Services Offered
    • Job Offerings
  • General Discussion
    • PHPFreaks.com Website Feedback
    • Miscellaneous

Find results in...

Find results that contain...


Date Created

  • Start

    End

Last Updated

  • Start

    End

Filter by number of...

Joined

  • Start

    End

Group

AIM

MSN

Website URL

ICQ

Yahoo

Jabber

Skype

Location

Interests

Age

Donation Link

Found 1 result

  1. lilmer
    I have a robots.txt on the site root directory with the content text of: User-agent: * Disallow: / By reading some stuffs they say that no crawler will engage the file content of my site like google and it will be safe for the bad crawlers not see the directory of your site and of course it is good for the security. But when I search the site on google it happen that A description for this result is not available because of this site's robots.txt – learn more. So my question now is how can a search engine be known the description of the site if your not allowing they're crawler not to engage the protected directory where all your file is ?
×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.