Batch processing SOAP in PHP5

[jonathanmusto]

Hi All,

 

First post so go easy!  I have a little problem and I wondered if anyone had done something similar before.

 

I'd like to batch process SOAP requests in PHP5 from a user uploaded .csv file.  I've done all the coding for the soap class and that works fine, I'm now just looking for a suitable solution to batching jobs throught it.

 

Essentially, a user will upload a .csv file in a pre defined format, this may contain 5 rows, it may contain 5,000.  Once the file has uploaded, id like the php script to batch each row through my SOAP class, storing the results for each call in an output .csv and, if possible, output some sort of percentage complete indicator to the user.  My current way of doing things is very clunky, doesn't give any user interaction and I had to increase the script timeout in php.ini to get it to work.... which i no aint good!!  It just uses fgetcsv to iterate through each row in realtime.

 

What would be the best way to go about this from a coding perspective?  AJAX?

 

I'm runing PHP on Windows by the way (horrible i know!)

 

Thanks,

J

 

 

[aschk]

I would say there probably is no other method. Not knowing the ins and outs of SOAP however, I would question whether you can build a large SOAP request (as it is essentially XML), and then rebuild the response into a CSV file by using simplexml?

 

e.g. request -

<xml>

  <node>"csv line 1 object"</node>

  <node>"csv line 2 object"</node>

  <node>"csv line 3 object"</node>

  <node>"csv line 4 object"</node>

</xml>

 

then send...

 

instead of

<xml>

  <node>"csv line 1 object"</node>

</xml>

send...

<xml>

  <node>"csv line 2 object"</node>

</xml>

send...

<xml>

  <node>"csv line 3 object"</node>

</xml>

send...

 

which is what i expect you are doing currently with each row.

 

It really does depend on the end points API that you're sending requests to. If it allows multiple nodes then my suggested method will work, otherwise you're stuck. Changing to AJAX isn't even an option (as far as I can see) as you have to upload the file to the server before anything can happen... the only other thing would be a fork a child process to deal with the batch scripting (which incidently is impossible in php at the current time, to the best of my knowledge), return a response page, which has AJAX that calls back to the server for the state of the child process (i.e. has it finished), and if it has finished requesting the completed CSV file.

 

Anyway, in summary, see my 1st suggestion...

[jonathanmusto]

Hi,

 

Thanks for your comprehensive reply, much appreciated.  Unfortunately the API i'm sending requests to only allows one node.  Essentially its a name and address search so looks as follows:

 

<applicant>
						<name>
							<title>MR</title>
							<forename>JOE</forename>
							<surname>BLOGGS</surname>
						</name>
						<currentaddress>
							<premiseno>37</premiseno>
							<postcode>X9 9XX</postcode>
							<addresstype>short</addresstype>
						</currentaddress>
						<dob>1980-01-06</dob>
					</applicant>

 

The CSV contains names and addresses, the script will have to do one SOAP call per row. 

 

I think your later option may be the only way forward for me.  Perhaps i could do the following:

 

1. User uploads file to the server using a html form.

2. Once upload is complete the user is taken to a page that displays a notification telling them the file is being processed.  This page could call the batch.php after flushing to the browser, which then processes the .csv file and performs the soap in the backround.

3. They then pick the file up once complete.

 

Thanks,

J

 

Thanks,

Jonathan

[aschk]

No output is sent to the browser until the PHP script has ended, so i'm not sure how you're planning on flushing the output to the browser and the have the PHP script continue to run. Let me know if you fashion a solution to this, and post how you did it here...