Jump to content

Google craw.


conan318

Recommended Posts

No, Google has absolutely no direct access to your database. Why would it want to? The data in there would be a jumbled up mess in the eyes of Google, even they would never be able to meaningfully index it -- never mind somehow guess the corresponding web-page to link it to. That leads to why semantic mark-up is so crucial in SEO.

 

Yellow Pages will use a single page, but pass in dynamic URL parameters to tell the code behind it what content to retrieve from the database and display on the page. This means that each entry will have it's own URL, but still point to the same file. I imagine they also take it a step further, and use the Apache "mod_rewrite" module (or equivalent for Windows) to create rewrite rules so that they can structure the URIs how they please.

 

So for example, you may request:

 

yellowpages.com/pizza/sheffield/pizza-hut

 

Which then the rewrite rules internally re-format into:

 

yellowpages.com/index.php?category=pizza&location=sheffield&brand=pizza-hut

 

The code within "index.php" would then handle retrieving and displaying the output, based upon the parameters passed in. I highly doubt Yellow Pages is this simple however, or even written in PHP, but hopefully you get the picture.

Link to comment
https://forums.phpfreaks.com/topic/236958-google-craw/#findComment-1217997
Share on other sites

Basically how adam said.

 

The crawling and your links can be gotten in a few ways.

 

A crawler can rip every link from your pages, (besides ajax), then follow every of those links to any depth it wants.

Generating random words and or pages in urls can be used in crawlers to try to get results from a website, could even be in a search form.

 

Can also get information from a sitemap if you have one.

They even acquire links from your site other sites they crawled that had a link of yours.

 

There's also feed crawlers grabbing latest posts.

Link to comment
https://forums.phpfreaks.com/topic/236958-google-craw/#findComment-1217999
Share on other sites

Archived

This topic is now archived and is closed to further replies.

×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.