Google craw.

[conan318]

I am planning to make a business index. i am wondering if anyone knows if google will craw your data base and index them?

 

thanks

[spiderwell]

it will crawl pages that exist, it wont get inside your database

[conan318]

so hows does placesy like yellowpages.com.au do it surely they don't make a page for every listing

[Adam]

No, Google has absolutely no direct access to your database. Why would it want to? The data in there would be a jumbled up mess in the eyes of Google, even they would never be able to meaningfully index it -- never mind somehow guess the corresponding web-page to link it to. That leads to why semantic mark-up is so crucial in SEO.

 

Yellow Pages will use a single page, but pass in dynamic URL parameters to tell the code behind it what content to retrieve from the database and display on the page. This means that each entry will have it's own URL, but still point to the same file. I imagine they also take it a step further, and use the Apache "mod_rewrite" module (or equivalent for Windows) to create rewrite rules so that they can structure the URIs how they please.

 

So for example, you may request:

 

yellowpages.com/pizza/sheffield/pizza-hut

 

Which then the rewrite rules internally re-format into:

 

yellowpages.com/index.php?category=pizza&location=sheffield&brand=pizza-hut

 

The code within "index.php" would then handle retrieving and displaying the output, based upon the parameters passed in. I highly doubt Yellow Pages is this simple however, or even written in PHP, but hopefully you get the picture.

[QuickOldCar]

Basically how adam said.

 

The crawling and your links can be gotten in a few ways.

 

A crawler can rip every link from your pages, (besides ajax), then follow every of those links to any depth it wants.

Generating random words and or pages in urls can be used in crawlers to try to get results from a website, could even be in a search form.

 

Can also get information from a sitemap if you have one.

They even acquire links from your site other sites they crawled that had a link of yours.

 

There's also feed crawlers grabbing latest posts.

[conan318]

thanks guys that helps me understand how it works  i am going to try and write something simple and see if i can get it to work will let you know how i go.