Mysql server hosting to support massive loads...

[ajlisowski]

Hey all. I am currently developing a project that could potentially need to support a mysql database which would be hit with millions of queries per minute.

 

Realistically, the load will likely never get that big, and if it did, I would first need to have a server capabale of handling thousands.

 

Right now I have a shared host which obviously wont cut it. I dont know that much about server management. I contacted a buddy of mine which once ran  his own webhost out of a rack in florida. I am waiting to hear back from him.

 

Every few seconds a user will be hitting the server with two queries and then updating a row. I can likely do some query caching to help aleviate, but the nature of the project means that most queries will be unique.

 

Will a dedicated server handle a few hundred users doing this at once? What about a few thousand? Obviously beyond that, if millions of folks are using it, id likely have to hire someone to manage the servers and run it ourselves, correct?

 

Or do companies exist which handle this kind of scaling from minimal use all the way up to millions of queries at once?

[QuickOldCar]

There is gonna be limits on how many you can actually do depending on the scripts you run, cpu usage, memory, amount of traffic and then your actual bandwidth.

 

Not to mention most likely tweaking apache to become more efficient.

 

A single dedicated server can do what you want. (shared sucks)

I really doubt "about a millions of queries per minute"

One of the above I mentioned would most likely bottleneck you.

 

Or do companies exist which handle this kind of scaling from minimal use all the way up to millions of queries at once?

Sure if want to spend lots of $$.

 

 

With doing so many requests you will have to establish some sort of cache just to take the load off, even if they are unique, if it even saves one fetch to the database it will save you a lot.

 

As a note, myisam locks the table on updates, which means no fetching until that is completed.

This article explains some of the good and bads of MyISAM and InnoDB

http://tag1consulting.com/MySQL_Engines_MyISAM_vs_InnoDB

 

Silly me... I'm using myisam because I wanted the full text indexing (I most likely will use cassandra in the future).

 

And while on the subject, I'll give a link for cassandra

http://cassandra.apache.org/

Cassandra is in use at Digg, Facebook, Twitter, Reddit, Rackspace, Cloudkick, Cisco, SimpleGeo, Ooyala, OpenX, and more companies that have large, active data sets. The largest production cluster has over 100 TB of data in over 150 machines.

 

 

As can see it's just not that easy to display lots of different data to many people at the same time without having lots of hardware and a proper database designed for server clusters.

[fenway]

Do you really have thousands of dollars per month in your budget for hosting?