RibTime Posted November 10, 2006 Share Posted November 10, 2006 I have a 82meg XML file that I want to process into an mySQL database so that I can use SQL to analyse it. I have some php using "simplexml" to load the file and parse out the data into an "INSERT" statement. The XML file is fine as I have some VB that processes it and that works fine.I know that I should end up with just over 87,000 records in the database, each with 34 fields.When I run the script it stops after about 2000 "inserts" with no error. If I cut down the file to approx 12meg I end up with about 8,500 records in the database.One of my thoughts is that I have a memory problem but lack of any message means I can't be sure.Does anyone have any advice as to what "logging" I can turn on to see what might be causing the problem.Thanks in advance. Link to comment https://forums.phpfreaks.com/topic/26827-problem-procesing-large-xml-file/ Share on other sites More sharing options...
bqallover Posted November 10, 2006 Share Posted November 10, 2006 You might want to have a look at [url=http://uk.php.net/set_time_limit]http://uk.php.net/set_time_limit[/url] to sort out your problem generally, by allowing the script to run for longer.Regarding logging, I couldn't really find anything helpful, though [url=http://xdebug.org/]xdebug[/url], a PECL extension, might serve your purpose as it will provide information on memory allocation.HTH :) Link to comment https://forums.phpfreaks.com/topic/26827-problem-procesing-large-xml-file/#findComment-122714 Share on other sites More sharing options...
RibTime Posted November 11, 2006 Author Share Posted November 11, 2006 Thank's for the help.I have already extended the time limit. When it fails you get a message that is pretty self describing.I'll check out the extension though.Many thanks Link to comment https://forums.phpfreaks.com/topic/26827-problem-procesing-large-xml-file/#findComment-123058 Share on other sites More sharing options...
Recommended Posts
Archived
This topic is now archived and is closed to further replies.