Jump to content

Web robots


otuatail

Recommended Posts

I wanted to get my website indexed by robots, but having seen some articles on the website this looks unlikely. According to some information some engines do not look at meta tags only the content. They avoid sites that have re-directions and cannot handle sites with a database backend as they cant access the data. This looks like they can see the WebPages at source on the server, not as pure html. They prefer to see the information at the top of the page. I have php at the top of mine. I have a session variable so I can create a hit counter, a simple redirection to under-construction so I can do development. A called function, to see if it is a robot and avoid a hit count. On top off all this I collect data to see which page has been visited. It appears that all this PHP is getting in the way. If I want my site indexed I have to strip out all code.

I have been asked to do a business site but don’t think I can get it on a search engine. Is there any way around this, or do I do strait html.


Paul.
Link to comment
Share on other sites

Web crawlers are pretty much like automated browsers. They see what we see.  Though they pick up on META tags and keywords etc within the content of the website.

They cannot see the source code, they can't see PHP/ASP, just like a user cannot see your source code.

There is talk about how the crawlers don't like to index pages with variables in the URL ie: http://blah.com/script.php?id=asd&view=thisplace but I've seen lots of pages that are indexed that way with google and yahoo, so they must do it, they might just perfer the http://blah.com/script.php/asd/thisplace better.

Submit your site to the big search engines.  Make sure there is alot of content in the pages you provide. Also make sure things images etc have title/alt tags associated to them too. Also ensure the content on your pages actual makes sense to the rest of the site.  Google likes to stop indexing your site if you use words that are not relevent to the rest of the site. IE having and white background and below your footer adding white text with popular search tems like teen, sex, etc when your site is about flowers ;)

Link to comment
Share on other sites

But a statment made said if they see a re-direction they will refuse because they think they are being conned. If they can't see php they can't see the re-direction.

If i wanted to force there hand i could (Knowing they are robots) deliver a difrent page to them and they wouldent know. (A page with cmore compresed information for them to see.)
Link to comment
Share on other sites

"If they can't see php they can't see the re-direction"

Yes, they can.  A redirection, perhaps with header("Location: index.php") for example, sends a 302 HTTP Response Code to the browser with details for the redirect if I remember correctly.  Therefore the browser can see it, meaning the robots can also.

Dest
Link to comment
Share on other sites

This thread is more than a year old. Please don't revive it unless you have something important to add.

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.