hamza Posted August 24, 2012 Share Posted August 24, 2012 please see me robot file # global Disallow: /cgi-bin/ Disallow: /wp-admin/ Disallow: /wp-includes/ Disallow: /wp-content/plugins/ Disallow: /wp-content/cache/ Disallow: /wp-content/themes/ Disallow: /trackback/ Disallow: /feed/ Disallow: /comments/ Disallow: /category/*/* Disallow: */trackback/ Disallow: */feed/ Disallow: */comments/ Disallow: /*? Allow: /wp-content/uploads/ #And here it is my subdir # Disallow Other subdirs Disallow: /test/ Disallow: /livelimmitless/ Disallow: /demos/ The Issue robots not crawlering on my site. and not indexing it . please see the file and help me with this what excactly the issue is. thanks Link to comment https://forums.phpfreaks.com/topic/267536-robottxt/ Share on other sites More sharing options...
ManiacDan Posted August 24, 2012 Share Posted August 24, 2012 This disallows almost every page of a WP install. Specifically this line: Disallow: /*? Unless you have mod_rewrite enabled, this blocks everything. What do the URLs to your site look like? Link to comment https://forums.phpfreaks.com/topic/267536-robottxt/#findComment-1372212 Share on other sites More sharing options...
Recommended Posts
Archived
This topic is now archived and is closed to further replies.