Run two php scripts (AJAX) to show loading

[iSE]

???

 

This is my first post on these forums so hello to all, I've been looking for a place to stick around and help where I can, but for now I'm in need of assistance...

 

I'm building a site with a content management system. In the admin area there is an option to upload a csv file. When you upload the csv file, the script runs through (normally around 550 records) each row, and parses the information. During this process it reads information from a HUGE MSSQL database, and then a combined recordset of the MSSQL data and the csv row data are inserted into a MySQL database which the site looks from.

 

Now this process needs to be performed every day and only after the user has uploaded a file which could be any time morning noon or night. At the moment, all runs fine although it can take around 5-20 minutes for the script to complete. Now it can be very annoying to wait around for that time, not knowing when it will complete (there may be a need to perform some tweaks manually after processing). What I'd like is to be able to show the progress of the parsing. This is the method I've been playing with so far...

 

Step 1. User uploads file and assuming it validates, is taken back to be shown a confirmation message.

Step 2. The confirmation message is shown and a javascript file is included.

Step 3. The javascript file uses AJAX to load the script to read the csv file and parse it.

 

    public function importCSV()
    {
        // Load the csv file as an object
        $csv = new Misc_ReadCSV(self::CSV_PATH . 'kerridge.csv', true);

        // Read the contents
        $ouput = $csv->fetch();

        // Set the number of rows in the csv file
        $_SESSION['kerridge']['totalrows'] = count($output);

        // Loop through each row and parse
        for ($i = 0; $i < count($ouput); $i++) {
            set_time_limit(30);

            // Call the parseVehicle method giving the data from the csv row
            if ($this->parseVehicle($ouput[$i]) === false) {
                throw new Exception('Error processing CSV file.');
            }
            
            // Set the row number being worked on
            $_SESSION['kerridge']['currentrow'] = $i;
        }

        return true;
    }

 

The idea here is that two session variables are set, the total number of rows is set at the beginning, and then inside the loop, $_SESSION['kerridge']['currentrow'] is updated to show the current row in the csv file being processed. And this brings me nicely to...

 

Step 4. A second AJAX call is made every n seconds (set to 10 for the moment) to run the script which echo's the results into the appropriate div.

 

if ($_SESSION['kerridge']['currentrow'] < $_SESSION['kerridge']['totalrows']) {
    echo '<p>' . $_SESSION['kerridge']['currentrow'] . ' / ' . $_SESSION['kerridge']['totalrows'];
} else {
    echo '<p>The used car stock database has been successfully uploaded.</p>';
}

 

Herein lies the problem... All I'm getting is:

 

Notice: Undefined variable: _SESSION in D:\inetpub\wwwroot\inc\ajax\kerridgestatus.php on line 2

 

Notice: Undefined variable: _SESSION in D:\inetpub\wwwroot\inc\ajax\kerridgestatus.php on line 2

 

The used car stock database has been successfully uploaded.

 

If anyone can help me with this, or knows of a better method to do what I'm asking, I thank you for your suggestions.

 

Kind Regards,

 

iSE

[kickstart]

Hi

 

Assuming the session has been started correctly it would appear OK. However I do not know if php will try and lock a session when one script is using it (preventing any other script from having the same session), but would think it unlikely.

 

I would be tempted to to a quick debug modification to your 2nd script to loop through and echo out the values of the $_SESSION array.

 

Failing that put a noddy table in place to hold the current and max record number for an upload and update that for each read row / read it for each check. Not particularly efficient, although you could improve it (only update every X rows, or just dump it to a flat file rather than a table).

 

All the best

 

Keith

[Jibberish]

PHP has a locking feature with sessions, so that it wont let the same session run two different scripts (which access the session) exicute at the same time.

 

Found this out on a recent project doing ajax call's, where the first on would run a report and the second would keep checking in the background to see if its done, however the second one would just hang untill the 1st completed.

 

saying that this looks like the session isn't being started correctly. are you calling session_start() on the second script?

[kickstart]

PHP has a locking feature with sessions, so that it wont let the same session run two different scripts (which access the session) exicute at the same time.

 

Interesting. Thankyou.

 

All the best

 

Keith

[kickstart]

Hi

 

Looks like you might be forced to free and reopen the sessions. It does seem that having closed a session for writing you can still read it, so possibly the AJAX check script could do a session start and immediatly a session_write_close.

 

Not sure how much of an overhead this is:-

 

    public function importCSV()
    {
        // Load the csv file as an object
        $csv = new Misc_ReadCSV(self::CSV_PATH . 'kerridge.csv', true);

        // Read the contents
        $ouput = $csv->fetch();

        // Set the number of rows in the csv file
        $_SESSION['kerridge']['totalrows'] = count($output);

        session_write_close();
        // Loop through each row and parse
        for ($i = 0; $i < count($ouput); $i++) {
            set_time_limit(30);

            // Call the parseVehicle method giving the data from the csv row
            if ($this->parseVehicle($ouput[$i]) === false) {
                throw new Exception('Error processing CSV file.');
            }
            
            // Set the row number being worked on
            if ($i % 10 == 0)
            {
                session_start();
                $_SESSION['kerridge']['currentrow'] = $i;
                session_write_close();
            }
        }

        session_start();
        return true;
    }

 

All the best

 

Keith

[iSE]

PHP has a locking feature with sessions, so that it wont let the same session run two different scripts (which access the session) exicute at the same time.

That will be the problem.

saying that this looks like the session isn't being started correctly. are you calling session_start() on the second script?

Yes I noticed that, and corrected the issue. This prevented the Notice that _SESSION is an undefined variable. Now however, I'm stuck with my check status script being unable to read the session variables.

 

Looks like you might be forced to free and reopen the sessions. It does seem that having closed a session for writing you can still read it, so possibly the AJAX check script could do a session start and immediatly a session_write_close.

I will give this a try and report back. Though I may need to pass the correct Session ID to the AJAX script somehow. The only way I can think of would be to echo it into a hidden input field, and then pass the value through post using JS. Could this be a security issue?

 

Thank you all for your help. At the moment, I've just left

ignore_user_abort(1);

so that the user can continue navigating with the script running in the background. This is an interesting challenge to say the least. Has it ever been done before using PHP and AJAX?

 

As for overheads, as always since the site is in testing, besides, even though the main site will take 5000+ hits a day, this is part of the administration area so should be run be a handful of people.

[kickstart]

Hi

 

I just used pretty much the same idea on a page (generating a large table of data from ~1000 separate databases).

 

I have managed it using session_start / session_write_close.

 

Worked fine in Firefox but not in IE. Seems there might be a limit in the number of times that you can do the session_start / session_write_close in IE, and if you exceed that it seemed to trigger a status of 8 when you returned the info in Ajax.

 

Now working by limiting how often it it does the session_start / session_write_close (from quick trials, about 100 seems to work).

 

All the best

 

Keith

[iSE]

Hey kickstart,

 

If you manage it, let me know! I managed to cut down the time during which the parsing takes place, showing just a simple loading animation until complete. Not ideal, but it'll do for now. This is an interesting challenge anyway and when I get some time to come back to it, I'll give it a proper go.

 

Thanks for your help,

 

Gary