Php search engine, How to deal with searching through alot of data?

[Joesavage]

I am working on a search function for my website. I have a single table which stores all of the text from each page on my website, among other things. So here is the problem, my site is huge. Assume that I add about 10000 rows a day to this table. And each row contains all of the text from a webpage, somewhere between 5,000 and 10,000 characters. So then when i search for a word I have too look at every description for every page and see if the description contains that word. So I am afraid that my current configuration will start to slow down very soon. Is there a better way to do this? I have looked at Ruby on Rails, im not sure if that will help, any thoughts? Also, is there a way to distribute the data over multiple databases/tables and if I do, will that speed anything up?  Please do not suggest a php search engine script.

[activeserver]

http://devzone.zend.com/node/view/id/91

Also, google: "zend search lucene"

 

No other popular(==supported) option that I know of.

 

if you can separate the indexing from the server that does the main stuff.... that will help a lot.

But that's not practical(bandwidth, https) or secure(http), IMO, on a shared host.