Jump to content

Problem Procesing Large XML File


RibTime

Recommended Posts

I have a 82meg XML file that I want to process into an mySQL database so that I can use SQL to analyse it. I have some php using "simplexml" to load the file and parse out the data into an "INSERT" statement. The XML file is fine as I have some VB that processes it and that works fine.

I know that I should end up with just over 87,000 records in the database, each with 34 fields.

When I run the script it stops after about 2000 "inserts" with no error. If I cut down the file to approx 12meg I end up with about 8,500 records in the database.

One of my thoughts is that I have a memory problem but lack of any message means I can't be sure.

Does anyone have any advice as to what "logging" I can turn on to see what might be causing the problem.

Thanks in advance.
Link to comment
https://forums.phpfreaks.com/topic/26827-problem-procesing-large-xml-file/
Share on other sites

You might want to have a look at [url=http://uk.php.net/set_time_limit]http://uk.php.net/set_time_limit[/url] to sort out your problem generally, by allowing the script to run for longer.

Regarding logging, I couldn't really find anything helpful, though [url=http://xdebug.org/]xdebug[/url], a PECL extension, might serve your purpose as it will provide information on memory allocation.

HTH :)

Archived

This topic is now archived and is closed to further replies.

×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.