kristo5747 Posted February 15, 2011 Share Posted February 15, 2011 Is there such thing? I designed a while back a rudimentary form based app for my users. We receive from our suppliers hardware manufacturing data in XML files: file name is made of eleven fields separated by tildes, with each field having its own meaning. R&D guys wanted to be able to search each field of the file names so I used regex() with decent results. Problem is that we have now in the upwards of 2.5 million files. And my app can't hack it anymore. I looked at Apache Lucene & Solr. Though it seemed like the best solution to my problem, the fields in the filenames are not peers to the file content. Big no-no with Solr. What is the best way to implement a PHP app with indexing and search capability with such large number of files? Do I have to buy Zend and use Zend_Search? Is it the only way? Thanks for your input. Quote Link to comment https://forums.phpfreaks.com/topic/227804-php-based-indexing-and-search-implementation/ Share on other sites More sharing options...
lastkarrde Posted February 16, 2011 Share Posted February 16, 2011 At the very least you will want to put that data into a database (MySQL, MongoDB, Postgres), searching through the file is horribly inefficient. You could then attempt to use the database's built in full text search. Best bet for performance and accuracy is to use something 3rd party like Lucene. Quote Link to comment https://forums.phpfreaks.com/topic/227804-php-based-indexing-and-search-implementation/#findComment-1174743 Share on other sites More sharing options...
QuickOldCar Posted February 16, 2011 Share Posted February 16, 2011 I second using a database. Then trying one of the following methods http://sphinxsearch.com/ http://lucene.apache.org/ http://www.zackgrossbart.com/hackito/search-engine-python/ Or even use mysql full text indexing http://dev.mysql.com/doc/refman/5.5/en/fulltext-search.html Quote Link to comment https://forums.phpfreaks.com/topic/227804-php-based-indexing-and-search-implementation/#findComment-1174840 Share on other sites More sharing options...
kristo5747 Posted February 16, 2011 Author Share Posted February 16, 2011 mysql full text indexing seems the way to go, alongside with Lucene. Thanks for your input. Quote Link to comment https://forums.phpfreaks.com/topic/227804-php-based-indexing-and-search-implementation/#findComment-1175138 Share on other sites More sharing options...
ignace Posted February 17, 2011 Share Posted February 17, 2011 Do I have to buy Zend and use Zend_Search? Zend_Search (as in Zend framework?) is free to use as is the entire framework. Quote Link to comment https://forums.phpfreaks.com/topic/227804-php-based-indexing-and-search-implementation/#findComment-1175525 Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.