hackalive Posted June 9, 2010 Share Posted June 9, 2010 Thanks to ignace and MrAdam I have managed to complete my web crawler script for a seach engine. Considering the scale this may grow to I need a clean and efficient DB design, so I am asking how you guys would do it. It would need somewhere for all the keywords and be able to link them to certain sites One table (tbl_site) might work like this |id|url|title|description Just looking for your guys opinion and how you think this DB could/may/should work Thanks in advance Link to comment https://forums.phpfreaks.com/topic/204240-data-storagre-for-seach-engine/ Share on other sites More sharing options...
hackalive Posted June 10, 2010 Author Share Posted June 10, 2010 bump Link to comment https://forums.phpfreaks.com/topic/204240-data-storagre-for-seach-engine/#findComment-1070286 Share on other sites More sharing options...
hackalive Posted June 12, 2010 Author Share Posted June 12, 2010 bump Link to comment https://forums.phpfreaks.com/topic/204240-data-storagre-for-seach-engine/#findComment-1071089 Share on other sites More sharing options...
ignace Posted June 12, 2010 Share Posted June 12, 2010 I don't think SE use a traditional DB like MySQL maybe something like NoSQL? Link to comment https://forums.phpfreaks.com/topic/204240-data-storagre-for-seach-engine/#findComment-1071099 Share on other sites More sharing options...
hackalive Posted June 12, 2010 Author Share Posted June 12, 2010 okay, i'll take a look thanks ignace Link to comment https://forums.phpfreaks.com/topic/204240-data-storagre-for-seach-engine/#findComment-1071108 Share on other sites More sharing options...
Mchl Posted June 12, 2010 Share Posted June 12, 2010 Google for example has developed it's own storage engine which is used in many of their services: http://en.wikipedia.org/wiki/BigTable Link to comment https://forums.phpfreaks.com/topic/204240-data-storagre-for-seach-engine/#findComment-1071115 Share on other sites More sharing options...
hackalive Posted June 12, 2010 Author Share Posted June 12, 2010 okay so if I was to use cassandra (for example) how would I organise the tables do you guys think? Thanks for the help so far Link to comment https://forums.phpfreaks.com/topic/204240-data-storagre-for-seach-engine/#findComment-1071125 Share on other sites More sharing options...
Recommended Posts
Archived
This topic is now archived and is closed to further replies.