CSV export database dump - memory error

[_tina_]

Hi,

 

I'm trying to export to a csv/xls file.  The user clicks a link, which calls a function that queries the database and prompts them with a download dialog.

 

I have an array of results, everything works fine, BUT, when I have 8000+ results, I get a PHP memory error.  Is there any way around this?

 

Thanks in advance!

 

[PFMaBiSmAd]

Any time you have an application that processes an ever increasing amount of data, you need to use memory management techniques (i.e. process the data in discrete chunks instead of all at once.)

 

It's also possible that your existing code is using 2 or 3 times more memory than necessary, but is would take seeing your existing code to be able to help with what it is doing or where memory management could be added to it.

[_tina_]

Thanks for the reply.

 

Here is what I am doing:

            $this->setLayout(false);

    $q = Doctrine_Query::create()
	  ->select('*')
	  ->from('ContestSubmission s')
	  ->leftJoin('Contest c ON s.id = c.contest_id')
	  ->where('s.contest_id = ?', $request->getParameter('id'));
	$this->contest = $q->execute(array(), Doctrine::HYDRATE_ARRAY);	 


    	$this->getResponse()->setContentType('application/msexcel');

 

This is done using a PHP framework btw.

 

Regarding memory management, what would you suggest I do to rectify this?

[PFMaBiSmAd]

Without information from you as to what framework you are using, no one can directly help.

 

Unless the framework provides a method to process the data in blocks, you will either need to write your own code to extend the framework or you will need to write your own code and not use a framework.

[_tina_]

I'm using Zend Framework, but this section is using straight PHP5, it's being migrated to ZF. 

 

So, this section, for now, will be done in straight PHP5. 

[_tina_]

Still haven't been able to resolve this.  I need, if a user clicks a link, a bd will be quieried and return send the rows to a csv file, which will then present the user with a doownload prompt.  The problem being that when there are 10,000+ records, I get a php memory error.  Is there a way around this at all?

 

Thanks for the replies so far and thanks for reading