Case is returning an Echo before the entire procedure is complete

[myrddinwylt]

Hello,

 

I am using AJAX to toggle back and forth between "switch/case" by sending a new POST var defining which "case" i wish to execute. At the end of each case, is an echo, which defines progress status, and essentialy is returned to the AJAX html request, and processed client side.

 

So far, it works great for any of the cases I have pushed on it, except this one.

 

        case 'merge_cdr':
            include("includes/ftp.class.php");
            include("includes/cdr.class.php");
            include("includes/http.class.php");
            include("includes/txt2sql.class.php");
            include("includes/mysql.class.php");

            $startdate = strtotime($download["startdate"]);

            $month = date("m",$startdate);
            $day = date("d",$startdate);
            $year = date("Y",$startdate);


            $daterange["start"] = mktime(0, 0, 0, $month ,$day,$year);
            $daterange["end"] = mktime(0,0,0,date("m"),date("d")-2,$year);

            $daysdownloaded = null;

            foreach($cdrformats as $cdrformat) {
                $cdr = new cdr();
                $cdr->dateposition = $cdrformat["date"];

                switch($cdrformat["type"]) {
                    case "ftp":
                        $ftp = new ftp();
                        $ftp->connect($cdrformat["server"],$cdrformat["username"],$cdrformat["password"],1);
                        $ftp->nlist($cdrformat["path"]);
                        $cfiles = $ftp->files($cdrformat["filter"]);
                        if($cdrformat["path"] != null && $cdrformat["path"] !="") {
                          $ftp->chdir($cdrformat["path"]);
                        }
                        foreach($cfiles as $file) {
                          $fdate = $cdr->filedate($file,$cdrformat["dateoffset"]);
                          if ($fdate >= $daterange["start"] && $fdate <= $daterange["end"]) {
                             if(file_exists($download['path'] . date("Y-m-d",$fdate) . "." . $cdrformat["name"]) || file_exists("sql\\" . date("Y-m",$fdate) . "\\" . date("Y-m-d",$fdate) . ".sql")) {
                               if($download['overwrite'] != 0) {
                                 set_time_limit(500);
                                 @unlink($download["path"] . $file);
                                 $ftp->get($file, $download['path'] . date("Y-m-d",$fdate) . "." . $cdrformat["name"]);
                                 $downloads[$cdrformat["name"]][] = $download['path'] . date("Y-m-d",$fdate) . "." . $cdrformat["name"];
                                 $daysdownloaded[] = date("Y-m-d",$fdate);
            //                     echo date("Y-m-d",$fdate) . " - " . $file . "<br>";
                               }
                             } else {
                               set_time_limit(500);
                               $ftp->get($file, $download['path'] . date("Y-m-d",$fdate) . "." . $cdrformat["name"]);
                               $downloads[$cdrformat["name"]][] = $download['path'] . date("Y-m-d",$fdate) . "." . $cdrformat["name"];
                               $daysdownloaded[] = date("Y-m-d",$fdate);
                             }
                          }
                        }
                        $ftp->disconnect();
                        $ftp = null;
                        break;

                    case "http" || "https":
                        $http = new http($cdrformat['server'],$cdrformat['username'],$cdrformat['password'],$cdrformat['type'] );
                        $http->nlist($cdrformat["path"],$cdrformat["prefixfile"]);
                        $cfiles = $http->files($cdrformat["filter"]);
                        foreach($cfiles as $file) {
                          $fdate = $cdr->filedate($file,$cdrformat["dateoffset"]);
                          if ($fdate >= $daterange["start"] && $fdate <= $daterange["end"]) {
                             if(file_exists($download['path'] . date("Y-m-d",$fdate) . "." . $cdrformat["name"]) || file_exists("sql\\" . date("Y-m",$fdate) . "\\" . date("Y-m-d",$fdate) . ".sql")) {
                               if($download['overwrite'] != 0) {
                                 set_time_limit(500);
                                 @unlink($download["path"] . $file);
                                 $http->get($file, $cdrformat["prefixfile"], $download['path'] . date("Y-m-d",$fdate) . "." . $cdrformat["name"]);
                                 $downloads[$cdrformat["name"]][] = $download['path'] . date("Y-m-d",$fdate) . "." . $cdrformat["name"];
                                 $daysdownloaded[] = date("Y-m-d",$fdate);
                               }
                             } else {
                               set_time_limit(500);
                               $http->get($file, $cdrformat["prefixfile"], $download['path'] . date("Y-m-d",$fdate) . "." . $cdrformat["name"]);
                               $downloads[$cdrformat["name"]][] = $download['path'] . date("Y-m-d",$fdate) . "." . $cdrformat["name"];
                               $daysdownloaded[] = date("Y-m-d",$fdate);
                             }
                          }
                        }
                        break;

                    default:
                        break;
                }
                $cdr = null;
            }

            $missingfiles = null;
            if($daysdownloaded != null) {
              $daysdownloaded = array_unique($daysdownloaded);
              foreach($daysdownloaded as $daydownloaded) {
                 foreach($cdrformats as $cdrformat) {
                   if(!file_exists("downloads\\" . $daydownloaded . "." . $cdrformat['name'])) { goto skipday; break; }
                 }

                 $mysqldb1 = new mysql($mysql['server'],$mysql['username'],$mysql['password'],$mysql['database'],$mysql['databasetmp'],$cdrformat['name']);

                 $databaseformat = "DROP TABLE IF EXISTS `##DATABASE##`.`##MONTH##`;\nCREATE TABLE `##DATABASE##`.`##MONTH##` ( `uniqueid` int(11) DEFAULT NULL, `Caller` varchar(255) DEFAULT NULL, `Started` datetime DEFAULT NULL, `Dialed` varchar(255) DEFAULT NULL, `DurationSec` int(11) DEFAULT NULL, `DurationMin` int(11) DEFAULT NULL, `Cost` double DEFAULT NULL, `Location` varchar(60) DEFAULT NULL, `Switch` varchar(15) DEFAULT 'Cellular');";
                 $databaseformat = str_replace("##MONTH##",substr($daydownloaded,0,-3),$databaseformat);
                 $databaseformat = str_replace("##DATABASE##",$mysql['databasetmp'],$databaseformat);
                 $mysqldb1->executesql($databaseformat);
                 $mysqldb1 = null;
                 unset($mysqldb1);
                 $i = 0;

                 foreach($cdrformats as $cdrformat) {
                   if(!file_exists("downloads\\" . $daydownloaded . "." . $cdrformat['name'])) { goto missingfile; }
                   $txt2sql = new txt2sql($cdrformat["name"],$cdrformat["delimiter"],$cdrformat['skiprows'],$mysql['databasetmp']);
                   $vals = $txt2sql->getFile($daydownloaded);
                   $txt2sql = null;
                   unset($txt2sql);

                   $b = "";
                   $c = "";
                   $d = "";

                   $b = str_replace("##DATABASE##",$mysql['databasetmp'],$cdrformat["sql"]["beforerates"]);

                   $d = str_replace("##DATABASE##",$mysql['databasetmp'],$cdrformat["sql"]["migrate"]) . "\n" . "DROP TABLE IF EXISTS `" . $mysql['databasetmp'] . "`.`" . $cdrformat['name'] . "`;\n";
                   $d = str_replace("##MONTH##",substr($daydownloaded,0,-3),$d);

                   $mysqldb = new mysql($mysql['server'],$mysql['username'],$mysql['password'],$mysql['database'],$mysql['databasetmp'],$cdrformat['name']);

                   if($cdrformat["userates"] == true) {
                     $c = implode("\n",$mysqldb->getrates($cdrformat['useratesfields']['ratefield'],$cdrformat['useratesfields']['locationfield'],$cdrformat['useratesfields']['durationfield'],$cdrformat['useratesfields']['codefield']));
                   }

                   $vals .= $b . $c . $d;
                   $b = null;
                   $c = null;
                   $d = null;

                   @mkdir("sql\\" . substr($daydownloaded,0,-3) . "\\" . substr($daydownloaded,-2),0777,1);
                   file_put_contents("sql\\" . substr($daydownloaded,0,-3) . "\\" . substr($daydownloaded,-2) . "\\" . $daydownloaded . "." . $cdrformat['name'] . ".sql", $vals);
                   $mysqldb->executesql($vals);

                   $vals = null;
                   $mysqldb = null;
                   unset($vals, $mysqldb, $b, $c, $d);
                   goto nextcdrformat;
missingfile:
                   $missingfiles[$cdrformat['name']][] = $daydownloaded;
                   $i++;
nextcdrformat:
                 }

                 foreach($cdrformats as $cdrformat) {
                   @unlink("downloads\\" . $daydownloaded . $cdrformat['name']);
                 }
                 if($i == 0) {
                   set_time_limit(3600);
                   $mysqldb1 = new mysql($mysql['server'],$mysql['username'],$mysql['password'],$mysql['database'],$mysql['databasetmp'],$cdrformat['name']);
                   $vals = implode("\n",$mysqldb1->dumpsql($daydownloaded));
                   @mkdir("sql\\" . substr($daydownloaded,0,-3),0777,1);
                   file_put_contents("sql\\" . substr($daydownloaded,0,-3) . "\\" . $daydownloaded . ".sql", $vals);
                   $mysqldb1->executesql("DROP TABLE IF EXISTS `" . $mysql['databasetmp'] . "`.`" . substr($daydownloaded,0,-3) . "`;");
                   $mysqldb1->executesql($vals);
                   $vals = null;
                   unset($vals, $mysqldb1);
//                   echo "MERGED: " . $daydownloaded . "<br />";
                 } else {
skipday:
//                   echo "MISSING: " . $daydownloaded . "<br />";
                 }

              }
            }
            echo "CDR's Downloaded & Merged";
            break;

 

This particular case returns the "echo" statement, while it is still processing -- in fact almost immediatly after AJAX makes the call for "merge_cdr".  I know that the PHP function is still processing as I can open up Navicat, and watch the tables and structures, and SQL changes occuring live --- After the "CDR's Downloaded & Merged" message has shown in the browser. This entire block of code takes approximatly 1-3 minutes per file, and there could be several files.

 

Any idea as to why the Echo is occuring before the code has completed processing ?

[myrddinwylt]

Anyone have an idea on this problem ?

[myrddinwylt]

Well..  resolved this one again on my own.  Turns out that PHP isn't quite as smart as other languages which support "GOTO".

 

        case 'merge_cdr':
            include("includes/ftp.class.php");
            include("includes/cdr.class.php");
            include("includes/http.class.php");
            include("includes/txt2sql.class.php");
            include("includes/mysql.class.php");

            $startdate = strtotime($download["startdate"]);

            $month = date("m",$startdate);
            $day = date("d",$startdate);
            $year = date("Y",$startdate);


            $daterange["start"] = mktime(0, 0, 0, $month ,$day,$year);
            $daterange["end"] = mktime(0,0,0,date("m"),date("d")-2,$year);

            $daysdownloaded = null;

            foreach($cdrformats as $cdrformat) {
                $cdr = new cdr();
                $cdr->dateposition = $cdrformat["date"];

                switch($cdrformat["type"]) {
                    case "ftp":
                        $ftp = new ftp();
                        $ftp->connect($cdrformat["server"],$cdrformat["username"],$cdrformat["password"],1);
                        $ftp->nlist($cdrformat["path"]);
                        $cfiles = $ftp->files($cdrformat["filter"]);
                        if($cdrformat["path"] != null && $cdrformat["path"] !="") {
                          $ftp->chdir($cdrformat["path"]);
                        }
                        foreach($cfiles as $file) {
                          $fdate = $cdr->filedate($file,$cdrformat["dateoffset"]);
                          if ($fdate >= $daterange["start"] && $fdate <= $daterange["end"]) {
                             if(file_exists($download['path'] . date("Y-m-d",$fdate) . "." . $cdrformat["name"]) || file_exists("sql\\" . date("Y-m",$fdate) . "\\" . date("Y-m-d",$fdate) . ".sql")) {
                               if($download['overwrite'] != 0) {
                                 set_time_limit(500);
                                 @unlink($download["path"] . $file);
                                 $ftp->get($file, $download['path'] . date("Y-m-d",$fdate) . "." . $cdrformat["name"]);
                                 $downloads[$cdrformat["name"]][] = $download['path'] . date("Y-m-d",$fdate) . "." . $cdrformat["name"];
                                 $daysdownloaded[] = date("Y-m-d",$fdate);
            //                     echo date("Y-m-d",$fdate) . " - " . $file . "<br>";
                               }
                             } else {
                               set_time_limit(500);
                               $ftp->get($file, $download['path'] . date("Y-m-d",$fdate) . "." . $cdrformat["name"]);
                               $downloads[$cdrformat["name"]][] = $download['path'] . date("Y-m-d",$fdate) . "." . $cdrformat["name"];
                               $daysdownloaded[] = date("Y-m-d",$fdate);
                             }
                          }
                        }
                        $ftp->disconnect();
                        $ftp = null;
                        break;

                    case "http" || "https":
                        $http = new http($cdrformat['server'],$cdrformat['username'],$cdrformat['password'],$cdrformat['type'] );
                        $http->nlist($cdrformat["path"],$cdrformat["prefixfile"]);
                        $cfiles = $http->files($cdrformat["filter"]);
                        foreach($cfiles as $file) {
                          $fdate = $cdr->filedate($file,$cdrformat["dateoffset"]);
                          if ($fdate >= $daterange["start"] && $fdate <= $daterange["end"]) {
                             if(file_exists($download['path'] . date("Y-m-d",$fdate) . "." . $cdrformat["name"]) || file_exists("sql\\" . date("Y-m",$fdate) . "\\" . date("Y-m-d",$fdate) . ".sql")) {
                               if($download['overwrite'] != 0) {
                                 set_time_limit(500);
                                 @unlink($download["path"] . $file);
                                 $http->get($file, $cdrformat["prefixfile"], $download['path'] . date("Y-m-d",$fdate) . "." . $cdrformat["name"]);
                                 $downloads[$cdrformat["name"]][] = $download['path'] . date("Y-m-d",$fdate) . "." . $cdrformat["name"];
                                 $daysdownloaded[] = date("Y-m-d",$fdate);
                               }
                             } else {
                               set_time_limit(500);
                               $http->get($file, $cdrformat["prefixfile"], $download['path'] . date("Y-m-d",$fdate) . "." . $cdrformat["name"]);
                               $downloads[$cdrformat["name"]][] = $download['path'] . date("Y-m-d",$fdate) . "." . $cdrformat["name"];
                               $daysdownloaded[] = date("Y-m-d",$fdate);
                             }
                          }
                        }
                        break;

                    default:
                        break;
                }
                $cdr = null;
            }

            $missingfiles = null;
            if($daysdownloaded != null) {
              $daysdownloaded = array_unique($daysdownloaded);
              foreach($daysdownloaded as $daydownloaded) {
                 foreach($cdrformats as $cdrformat) {
                   if(!file_exists("downloads\\" . $daydownloaded . "." . $cdrformat['name'])) { goto skipday; break; }
                 }

                 $mysqldb1 = new mysql($mysql['server'],$mysql['username'],$mysql['password'],$mysql['database'],$mysql['databasetmp'],$cdrformat['name']);

                 $databaseformat = "DROP TABLE IF EXISTS `##DATABASE##`.`##MONTH##`;\nCREATE TABLE `##DATABASE##`.`##MONTH##` ( `uniqueid` int(11) DEFAULT NULL, `Caller` varchar(255) DEFAULT NULL, `Started` datetime DEFAULT NULL, `Dialed` varchar(255) DEFAULT NULL, `DurationSec` int(11) DEFAULT NULL, `DurationMin` int(11) DEFAULT NULL, `Cost` double DEFAULT NULL, `Location` varchar(60) DEFAULT NULL, `Switch` varchar(15) DEFAULT 'Cellular');";
                 $databaseformat = str_replace("##MONTH##",substr($daydownloaded,0,-3),$databaseformat);
                 $databaseformat = str_replace("##DATABASE##",$mysql['databasetmp'],$databaseformat);
                 $mysqldb1->executesql($databaseformat);
                 $mysqldb1 = null;
                 unset($mysqldb1);
                 $i = 0;

                 foreach($cdrformats as $cdrformat) {
                   if(!file_exists("downloads\\" . $daydownloaded . "." . $cdrformat['name'])) { goto missingfile; }
                   $txt2sql = new txt2sql($cdrformat["name"],$cdrformat["delimiter"],$cdrformat['skiprows'],$mysql['databasetmp']);
                   $vals = $txt2sql->getFile($daydownloaded);
                   $txt2sql = null;
                   unset($txt2sql);

                   $b = "";
                   $c = "";
                   $d = "";

                   $b = str_replace("##DATABASE##",$mysql['databasetmp'],$cdrformat["sql"]["beforerates"]);

                   $d = str_replace("##DATABASE##",$mysql['databasetmp'],$cdrformat["sql"]["migrate"]) . "\n" . "DROP TABLE IF EXISTS `" . $mysql['databasetmp'] . "`.`" . $cdrformat['name'] . "`;\n";
                   $d = str_replace("##MONTH##",substr($daydownloaded,0,-3),$d);

                   $mysqldb = new mysql($mysql['server'],$mysql['username'],$mysql['password'],$mysql['database'],$mysql['databasetmp'],$cdrformat['name']);

                   if($cdrformat["userates"] == true) {
                     $c = implode("\n",$mysqldb->getrates($cdrformat['useratesfields']['ratefield'],$cdrformat['useratesfields']['locationfield'],$cdrformat['useratesfields']['durationfield'],$cdrformat['useratesfields']['codefield']));
                   }

                   $vals .= $b . $c . $d;
                   $b = null;
                   $c = null;
                   $d = null;

                   @mkdir("sql\\" . substr($daydownloaded,0,-3) . "\\" . substr($daydownloaded,-2),0777,1);
                   file_put_contents("sql\\" . substr($daydownloaded,0,-3) . "\\" . substr($daydownloaded,-2) . "\\" . $daydownloaded . "." . $cdrformat['name'] . ".sql", $vals);
                   $mysqldb->executesql($vals);

                   $vals = null;
                   $mysqldb = null;
                   unset($vals, $mysqldb, $b, $c, $d);
                   goto nextcdrformat;
missingfile:
                   $missingfiles[$cdrformat['name']][] = $daydownloaded;
                   $i++;
nextcdrformat:
                 }

                 foreach($cdrformats as $cdrformat) {
                   @unlink("downloads\\" . $daydownloaded . $cdrformat['name']);
                 }
                 if($i == 0) {
                   set_time_limit(3600);
                   $mysqldb1 = new mysql($mysql['server'],$mysql['username'],$mysql['password'],$mysql['database'],$mysql['databasetmp'],$cdrformat['name']);
                   $vals = implode("\n",$mysqldb1->dumpsql($daydownloaded));
                   @mkdir("sql\\" . substr($daydownloaded,0,-3),0777,1);
                   file_put_contents("sql\\" . substr($daydownloaded,0,-3) . "\\" . $daydownloaded . ".sql", $vals);
                   $mysqldb1->executesql("DROP TABLE IF EXISTS `" . $mysql['databasetmp'] . "`.`" . substr($daydownloaded,0,-3) . "`;");
                   $mysqldb1->executesql($vals);
                   $vals = null;
                   unset($vals, $mysqldb1);
//                   echo "MERGED: " . $daydownloaded . "<br />";
                 } else {
//                   echo "MISSING: " . $daydownloaded . "<br />";
                 }
skipday:
              }
            }
            echo "CDR's Downloaded & Merged";
            break;

 

Had to move skipday:to just outside the last if statment.

[Adam]

Turns out that PHP isn't quite as smart as other languages which support "GOTO".

 

Wow, I don't think you realise how not-smart you just sounded with that allegation. I'm not going to go into what's wrong with GOTO statements, or try to understand how a GOTO statement would solve your problem?? But just to add insult to injury, check out the PHP GOTO statement, available as of PHP5.3.

 

To answer your original question; during long and/or memory intense loads like in your situation, PHP attempts to periodically output whatever is in the buffer. This is normally a positive thing in most situations as it free's up memory and resources. If you want to stop that you need to take control of the output buffer yourself.

[myrddinwylt]

First MrAdam,

 

What exactly is your point of providing a link to the following information ?  It doesn't state that it can not be used to point inside of an IF statement, only can not be pointed inside another loop or file.

The goto operator can be used to jump to another section in the program. The target point is specified by a label followed by a colon, and the instruction is given as goto followed by the desired target label. This is not a full unrestricted goto. The target label must be within the same file and context, meaning that you cannot jump out of a function or method, nor can you jump into one. You also cannot jump into any sort of loop or switch structure. You may jump out of these, and a common use is to use a goto  in place of a multi-level break.

 

Second, I wasn't asking for you to define how a GOTO statement would solve a problem. It does resolve a problem by making the code simpler, and more controllable without the use of extending IF statements to encompass the entire body or other similar logic. And NO, PHP does not have the same power of other languages when it comes to use of the GOTO command.  For example, in Visual Basic, you CAN use it to go inside another loop (which isn't what was being done here, but whatever).

 

Controlling output buffer is not really my concern. As stated in the original post, this code runs invisible, and does not output ANYTHING until an echo or print statement is made. Notice there is no where any data is to be output inside that block of code until the loop has finished.  PHP had no reason to leave that loop, until it's execution was completed, unless you are implying that PHP executes code in a random order, which would pretty much break every language rule out there.

 

The inherant and undocumented flaw in PHP as proven above, is as follows.

 

Breaking out of a foreach Loop using goto to enter into an else portion of an if statement causes PHP to continue execution past two closing }, regardless that the second } is the looping/closing brace for an outer loop.

 

So my not-smart allegation is more than just an "allegation". It is a fact. PHP's GOTO command is not quite as smart as other languages which support GOTO.

 

As for the Wikipedia entry on how GOTO is fail in Fortran, I couldn't give a flying --- .  It is still something used and does provide more control over your code without slopping it over using 300 closing braces for a simple IF check. As for further evidence of it's importance, it's inclusion in even the latest version of PHP (albeit limited, but at least they are trying and it does work -- mostly).

 

Perhaps this was a statement you missed in that Wiki entry,

The goto statement is often combined with the if statement to cause a conditional transfer of control.

, which should help to clarify it's purpose in development --- YOU get to take control of the code, instead of leaving it to the whim of the pairing elseif or else statements, which in large blocks of code tend to get shoveled several hundred lines down... or more.

 

In any case, thank you for your response, even though it was an attempt to insult my work/intelligence/coding techniques.

 

 

 

Turns out that PHP isn't quite as smart as other languages which support "GOTO".

 

Wow, I don't think you realise how not-smart you just sounded with that allegation. I'm not going to go into what's wrong with GOTO statements, or try to understand how a GOTO statement would solve your problem?? But just to add insult to injury, check out the PHP GOTO statement, available as of PHP5.3.

 

To answer your original question; during long and/or memory intense loads like in your situation, PHP attempts to periodically output whatever is in the buffer. This is normally a positive thing in most situations as it free's up memory and resources. If you want to stop that you need to take control of the output buffer yourself.