ursvmg Posted June 5, 2010 Share Posted June 5, 2010 Hi all, I have a PHP to scrap huge data(around 10000 entries) from a html page using simplehtmldom. My objective is to load the data in to MySQL table. I know, it is not a good idea to scrap and load at a stretch. Now I have been inserting 100 rows at a time, ending up with 100 insert queries for every html page. I would like to know how best I can handle this situation considering performance. What are the PHP/MySQL parameters that I need to set? Kindly throw some light on memory management in PHP. Regards, ursvmg Link to comment https://forums.phpfreaks.com/topic/203976-php-memory-management/ Share on other sites More sharing options...
GoneNowBye Posted June 5, 2010 Share Posted June 5, 2010 php_set_time(0); it may be set_time_limit...google it make sure you dont have any endless loops and unset() your vars after use you could also use set_ini('Memory_limit','512MB'); or the appropreate command Link to comment https://forums.phpfreaks.com/topic/203976-php-memory-management/#findComment-1068356 Share on other sites More sharing options...
Mchl Posted June 5, 2010 Share Posted June 5, 2010 Use extended inserts. Link to comment https://forums.phpfreaks.com/topic/203976-php-memory-management/#findComment-1068402 Share on other sites More sharing options...
ursvmg Posted June 6, 2010 Author Share Posted June 6, 2010 Thanks for your replies. Link to comment https://forums.phpfreaks.com/topic/203976-php-memory-management/#findComment-1068474 Share on other sites More sharing options...
Recommended Posts
Archived
This topic is now archived and is closed to further replies.