ursvmg Posted June 5, 2010 Share Posted June 5, 2010 Hi all, I have a PHP to scrap huge data(around 10000 entries) from a html page using simplehtmldom. My objective is to load the data in to MySQL table. I know, it is not a good idea to scrap and load at a stretch. Now I have been inserting 100 rows at a time, ending up with 100 insert queries for every html page. I would like to know how best I can handle this situation considering performance. What are the PHP/MySQL parameters that I need to set? Kindly throw some light on memory management in PHP. Regards, ursvmg Quote Link to comment https://forums.phpfreaks.com/topic/203976-php-memory-management/ Share on other sites More sharing options...
GoneNowBye Posted June 5, 2010 Share Posted June 5, 2010 php_set_time(0); it may be set_time_limit...google it make sure you dont have any endless loops and unset() your vars after use you could also use set_ini('Memory_limit','512MB'); or the appropreate command Quote Link to comment https://forums.phpfreaks.com/topic/203976-php-memory-management/#findComment-1068356 Share on other sites More sharing options...
Mchl Posted June 5, 2010 Share Posted June 5, 2010 Use extended inserts. Quote Link to comment https://forums.phpfreaks.com/topic/203976-php-memory-management/#findComment-1068402 Share on other sites More sharing options...
ursvmg Posted June 6, 2010 Author Share Posted June 6, 2010 Thanks for your replies. Quote Link to comment https://forums.phpfreaks.com/topic/203976-php-memory-management/#findComment-1068474 Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.