Several servers working with single MySQL database

[SuperDennis]

Hello,

 

I think of a big project, which gonna take much load on sever resources and bandwidth. It should work with huge MySQL database of different webpages (maybe about 20 million, I hope MySQL can handle such amount of records). But I will need to share the requests to the database between several servers I think, because one single server won't be able to handle such a load. I wonder how big projects like Google or Archive.org are doing this? I thought of primitive method, which is like follows:

[*]The MySQL database is located on an independent standalone server.

[*]The MySQL database table contains a field, say, "server", with particular server name, for which the record is allocated for processing.

[*]Each server connects MySQL server and works ONLY with records, which are allocated to that server (by looking into "server" field in that table).

 

I feel like this is a noob method, but I would like to know your opinion, and maybe the ways you see how this can be implemented in more optimized and clever form.

 

Will be happy for any tip.

 

Thanks,

Dennis.

[trq]

Firstly, 20 million records is not a large database as far as MySql is concerned.

 

Secondly, inventing your own half baked method of replication should be avoided completely. MySql has support for high availability and scalability. See http://dev.mysql.com/doc/refman/5.1/en/ha-overview.html

 

As for Google. They use there own database engine (BigTable) for most of there database work not MySql. There are open source (similarly designed) databases such as cassandra (developed, used and made OSS by facebook) but they are a completely different database model to that of MySql and use a NoSQL approach.

[Anti-Moronic]

Firstly, 20 million records is not a large database as far as MySql is concerned.

 

Secondly, inventing your own half baked method of replication should be avoided completely. MySql has support for high availability and scalability. See http://dev.mysql.com/doc/refman/5.1/en/ha-overview.html

 

As for Google. They use there own database engine (BigTable) for most of there database work not MySql. There are open source (similarly designed) databases such as cassandra (developed, used and made OSS by facebook) but they are a completely different database model to that of MySql and use a NoSQL approach.

 

Just thinking, if cassandra is open source why are more people not using it? Looks far superior to mysql. Love to hear your opinions! Have you started using it yet?

[ignace]

I think of a big project, which gonna take much load on sever resources and bandwidth.

 

Don't think, measure! There's a big difference between having and assuming you'll have 20 million records. In either case it won't hurt to dive into serious MySQL administration and learn it's different engines and their differences, indexes and how they work and how to find bottlenecks, replication, ..

 

High Performance MySQL and SQL anti-patterns should lead you through the maze!