an advanced question about saving sever resources SQL / Remote SQL, or CurL

[yanivkalfa]

Well my question is not 100% about MySQL or specific DB, but its very relevant to that and i didn't knew where to post it.

 

So, i have a network of website , 1 of the website control all of the content in those sites, the way i am operating the network now is, i have a script on every one of the sites in the network when this script is being visited by "cronjob" sorta script it connect using Curl to the main website request the data, and then insert it to the DB of that site.

 

I've found that this system is some what heavy on resources when i have around 150/200 websites.

 

My question is.

i have several options I've thought of to reduce the resources usage.

 

Option 1.) 2 main DB, DB (A) for the main website, DB(B) for all the other websites - instead of using Curl, the main website will remote connect to the DB(B) and insert or update the data, the only problem i can see with this method is 150-200 website all connection to that same DB every time a user visit them, can create a problem maybe ?.

Option 2.) not changing the DB structure, having 1 DB on main website and one DB for each website, but instead of using Curl making the main website remote connect to all the 150-200 website and insert/update the data as needed. problems with this method, well first its not much of a difference between the way i am using now to this so i don't know how much it will improve resources wise, i am pretty sure however that i will be a lot slower then the current system.

Option 3.) No use of DB at all, instead use files, and regular expression to extract / insert or update the data, so when one of the scripts are visited it will request the file from the main website, i don't know how this will workout simple because DB are much more accepted and normal and i don't know if anyone uses this kind of method.

 

any kind of help regarding this will be Highly appreciated and i know this aint a common questions.

 

Thanks in advanced.

 

[fenway]

I'm guessing the load is the curl part -- not sure what the db has to do with this.

[yanivkalfa]

well, i never thought its DB related, but the dedicated host we had kept going down and the company that provided it always claimed it was heavy DB resources consumption. you are saying its Curl which means using option A will be best.

 

I thank you for your answer.

[fenway]

I'm guessing -- depends how you're populating and pulling from that DB.

[yanivkalfa]

Well i guess i will have to try.. thanks for your time.