Read files in directory and put into a two Dimensional Array and read content

wheakory · August 27, 2009

I need to read a directory with files that have certain extensions (4 character extension prior the period . and number scheme) (.m1ad .m1ds, .m1en, .m1pg, .m1ps) into an two dimensional associate array in php. Then I need to read each one of the files contents from the array, because they contain google user create accounts, password changes, enables and disables of gmail accounts using php and ZendFrame Work.

This program will be running from the command line in the background (on a linux System), constantly picking up the new files to process the transactions. There could be like 20 (.m1ad or .m1en) files in that directory. That's why I thought putting them in an array would be easier, because the files in the directory needs to be deleted after the transactions in the file content is processed.

Current code that does reads one file in the directory

#!/usr/bin/php -q
<html>
     <head>
      <meta name=keywords content="Google Zenda data">
      <meta name=description content="Google Zenda Data test"> 
       <title>Google Zend Data Form</title>
    </head>
<body>
<?php
include("connect-gmail.inc");

      // load classes
      require_once 'Zend/Loader.php';
      Zend_Loader::loadClass('Zend_Gdata');
      Zend_Loader::loadClass('Zend_Gdata_ClientLogin');
      Zend_Loader::loadClass('Zend_Gdata_Gapps');
     //  Zend_Loader::loadClass('Zend_Gdata_Calendar');
      Zend_Loader::loadClass('Zend_Http_Client');
      
      // connect to service
      //  HERES WHERE I"M JUST READING ONE FILE FROM DIRECTORY RIGHT NOW     
    $filestuff = "/home/acctmnt/acctFiles/052809285196.m1ad";
      $user = "isuadmin@gdev.isu.edu";
      $pass = "S3rV1c3!";
      $domain = "gdev.isu.edu";
      $client = Zend_Gdata_ClientLogin::getHttpClient($user, $pass, Zend_Gdata_Gapps::AUTH_SERVICE_NAME);
      $service = new Zend_Gdata_Gapps($client, $domain);
      $error_flag = "no";
      $err_message = "";
      
    // read file
    $fp = fopen($filestuff, "r");
    while ($line = fgets($fp))
    {
    $username = strtok($line, ":");
    $familyname = strtok(":");
    $givenname = strtok(":");
    $passwordname = strtok(":");
     
      // construct event object
      // save to server      
      try {
        $quota = "50M";
        $service->createUser($username, $familyname, $givenname, $passwordname, $passwordHashFunction=null, $quota=null);         
        $query2 = "INSERT INTO gmail_add VALUES('$username', '$familyname', '$givenname', '$quota', NOW() )";
        mysql_query( $query2 ) or die(mysql_error());  
} 
catch (Zend_Gdata_Gapps_ServiceException $e) {
    if ($e->hasError(Zend_Gdata_Gapps_Error::ENTITY_EXISTS)) {
      // If this is an ENTITY_DOES_NOT_EXIST error, return null
        $err_message = $err_message . "Already EXISTS \r\n";
        $error_flag = "yes";
    }
    elseif ($e->hasError(Zend_Gdata_Gapps_Error::USER_DELETED_RECENTLY)) {
      // If this is an ENTITY_DOES_NOT_EXIST error, return null
         $err_message = $err_message . "Email Account Deleted Recently cannot create until 5 days \r\n";
         $error_flag = "yes";
    }
    elseif ($e->hasError(Zend_Gdata_Gapps_Error::ENTITY_NAME_IS_RESERVED)) {
      // If this is an ENTITY_RESERVED error, return null
         $err_message = $err_message . "Email Account Name Reserved \r\n";
         $error_flag = "yes";
    }
    else {
      // Outherwise, just print the errors that occured and exit
      foreach ($e->getErrors() as $error) {
        echo "Error encountered: {$error->getReason()} ({$error->getErrorCode()})\n";
      } // foreach
      exit();
    } // if ENTITY EXISTS
  } // Catch
        echo "user $error_flag \n>";
     if ($error_flag == "yes") {
         // define current time  
         $time = date('H:i:s');  
         // fwrite($this->fp, "$time ($script_name) $message\n");  
      
         // define the current date (it will be appended to the log file name)  
         $today = date('Y-m-d');  
         // open log file for writing only; place the file pointer at the end of the file  
      
         $file = fopen("/home/acctmnt/logs/acctlog.txt", 'a') or die("can't open file");
         fwrite($file,"$username:m1ad $today:$time $username $err_message \r\n");
         fclose($file);
         $error_flag = "no";
         $err_message = "";
     }else{
         // define current time  
         $time = date('H:i:s');  
         // define the current date (it will be appended to the log file name)  
         $today = date('Y-m-d');  
    
         $file = fopen("/home/acctmnt/logs/acctlog.txt", 'a') or die("can't open file");
         fwrite($file,"$username:m1ad $today:$time $username Email Account Successful \r\n");
         fclose($file);
         $error_flag = "no";
         $err_message = "";
     } // error flag
} //while loop
    ?>
</body>
</html>

lemmin · August 27, 2009

You can use glob() to put every file name into an array and then check your requirements. If this folder contains files other than the ones you want to parse then you would have to have some conditions before parsing them.

$files = glob("dir");
foreach ($files as $file)
{
   if (is_file($file) && [matches regex for filetype])
      parseFile($file);
}

The parseFile() function can be that code that you already have.

wheakory · August 27, 2009

You can use glob() to put every file name into an array and then check your requirements. If this folder contains files other than the ones you want to parse then you would have to have some conditions before parsing them.
$files = glob("dir");
foreach ($files as $file)
{
   if (is_file($file) && [matches regex for filetype])
      parseFile($file);
}
The parseFile() function can be that code that you already have.

I need to know what the extension is for each file because they will be sent to a different function in the script for processing. Example files will be 34434.m1ad 54dfddf.m1ad (add an email account) 34343.m1ds, 3443ddfd.m1ds (disable an email account) so how would I read these different extensions into different arrays and then process the array files for each extension all at once in the differents they will go to by the extension match.

lemmin · August 27, 2009

You can use a regular expression like this:

preg_match("/\.[^\.]+$/", $filename, $matches);
$ext = strtolower($matches[0]);

Then you can switch $ext based on the different known cases.

wheakory · August 27, 2009

You can use a regular expression like this:
preg_match("/\.[^\.]+$/", $filename, $matches);
$ext = strtolower($matches[0]);
Then you can switch $ext based on the different known cases.

Could you give me a code example of the whole process of moving it into the different extension arrays and then apply an if statement that would say if the array value equal this extension go to this function. This php type of code is new for me.

lemmin · August 27, 2009

I don't know exactly what you have to change per each file, but you can make the adjustments in a switch statement about the extension:

$files = glob("dir");
foreach ($files as $file)
{
   if (is_file($file) && [matches regex for filetype])
      parseFile($file);
}

parseFile($file)
{
   preg_match("/\.[^\.]+$/", $filename, $matches);
   $ext = strtolower($matches[0]);
   
   switch ($ext)
   {
      case ".m1ds":
         [code unique to this file type]
         break;
      case ".m1en":
         [code unique to this file type]
         break;
   }

    $fp = fopen($file, "r");
    while ($line = fgets($fp))
    {
       [...]
}

wheakory · August 27, 2009

I don't know exactly what you have to change per each file, but you can make the adjustments in a switch statement about the extension:

$files = glob("dir");
foreach ($files as $file)
{
   if (is_file($file) && [matches regex for filetype])
      parseFile($file);
}

parseFile($file)
{
   preg_match("/\.[^\.]+$/", $filename, $matches);
   $ext = strtolower($matches[0]);
   
   switch ($ext)
   {
      case ".m1ds":
         [code unique to this file type]
         break;
      case ".m1en":
         [code unique to this file type]
         break;
   }

    $fp = fopen($file, "r");
    while ($line = fgets($fp))
    {
       [...]
}

Here's basically what I need to do. The files will be ftp over as transaction and they will be in the directory unique like this (33fdfd.m1ad, 34343.m1pg, afdkljfa.m1en, 334343.m1ds) These are Create, purge, enable, and disable gmail transactions. The .m1ad files are create accounts. The .m1pg are purge accounts. The .m1en are Enable accounts, and .m1ds are disable accounts. I need to list these files from the directory (which shouldn't have any other file extensions than what I specified above), Check the file extension and put each one in its own separate array by file extension. Then read the array file content records and process the account by whatever transaction type it is (.m1ad, ,m1pg, .m1en, .m1ds). Does that make more sense. Thanks for your help.

lemmin · August 27, 2009

Yeah, that is pretty much what I figured. The code that I posted would work just fine for that. The part that I don't know is how much of the parse code needs to be different for each file type. One way to do it is to make a function for each operation you need to do: parsePurge(), parseEnable(), etc... Then you can just switch to those different functions like I showed:

$files = glob("dir");
foreach ($files as $file)
{
   preg_match("/\.[^\.]+$/", $filename, $matches);
   $ext = strtolower($matches[0]);
   switch ($ext)
   {
      case ".m1ds":
         parseDisable($file);
         break;
      case ".m1en":
         parseDisable($file);
         break;
      case ".m1ad":
         parseCreate($file);
         break;
      case ".m1pg":
         parsePurge($file);
         break;
      default:
         die("error");
   }
}

If you do it this way, you don't need an array for each file type because they are all parsed differently in one pass.

wheakory · August 27, 2009

The files look like this (and they may have more than one record)

(delimiter is a ":")

(filename) (record values delimited by a colon).

3333.m1ad tempp.gmail1:temp:google1:temp1234

3333.m1pg tempp.gmail1:

3333.m1en tempp.gmail1:

3333.m1ds tempp.gmail1:

So I could do something like this to read the file below (sorry for the stupid questions, but I'm looking for the most efficient method for the production code)

$files = glob("dir");
foreach ($files as $file)
{
   preg_match("/\.[^\.]+$/", $filename, $matches);
   $ext = strtolower($matches[0]);
   switch ($ext)
   {
      case ".m1ds":
         parseDisable($file);
         break;
      case ".m1en":
         parseEnable($file);
         break;
      case ".m1ad":
         parseCreate($file);
         break;
      case ".m1pg":
         parsePurge($file);
         break;
      default:
         die("error");
   }
}

parseCreate($file) {
/ /read file
    $fp = fopen($file, "r");
    while ($line = fgets($fp))
    {
      $username = strtok($line, ":");
     $familyname = strtok(":");
     $givenname = strtok(":");
     $passwordname = strtok(":");
/ /read file
    $fp = fopen($file, "r");
    while ($line = fgets($fp))
    {
      $username = strtok($line, ":");
    $familyname = strtok(":");
    $givenname = strtok(":");
    $passwordname = strtok(":");
   
      // construct event object
      // save to server      
      try {
        $quota = "50M";
        $service->createUser($username, $familyname, $givenname, $passwordname, $passwordHashFunction=null, $quota=null);         
        $query2 = "INSERT INTO gmail_add VALUES('$username', '$familyname', '$givenname', '$quota', NOW() )";
        mysql_query( $query2 ) or die(mysql_error());  
} 
catch (Zend_Gdata_Gapps_ServiceException $e) {
    if ($e->hasError(Zend_Gdata_Gapps_Error::ENTITY_EXISTS)) {
      // If this is an ENTITY_DOES_NOT_EXIST error, return null
        $err_message = $err_message . "Already EXISTS \r\n";
        $error_flag = "yes";
    }
    elseif ($e->hasError(Zend_Gdata_Gapps_Error::USER_DELETED_RECENTLY)) {
      // If this is an ENTITY_DOES_NOT_EXIST error, return null
         $err_message = $err_message . "Email Account Deleted Recently cannot create until 5 days \r\n";
         $error_flag = "yes";
    }
    elseif ($e->hasError(Zend_Gdata_Gapps_Error::ENTITY_NAME_IS_RESERVED)) {
      // If this is an ENTITY_RESERVED error, return null
         $err_message = $err_message . "Email Account Name Reserved \r\n";
         $error_flag = "yes";
    }
    else {
      // Outherwise, just print the errors that occured and exit
      foreach ($e->getErrors() as $error) {
        echo "Error encountered: {$error->getReason()} ({$error->getErrorCode()})\n";
      } // foreach
      exit();
    } // if ENTITY EXISTS
  } // Catch
        echo "user $error_flag \n>";
     if ($error_flag == "yes") {
         // define current time  
         $time = date('H:i:s');  
         // fwrite($this->fp, "$time ($script_name) $message\n");  
      
         // define the current date (it will be appended to the log file name)  
         $today = date('Y-m-d');  
         // open log file for writing only; place the file pointer at the end of the file  
      
         $file = fopen("/home/acctmnt/logs/acctlog.txt", 'a') or die("can't open file");
         fwrite($file,"$username:m1ad $today:$time $username $err_message \r\n");
         fclose($file);
         $error_flag = "no";
         $err_message = "";
     }else{
         // define current time  
         $time = date('H:i:s');  
         // define the current date (it will be appended to the log file name)  
         $today = date('Y-m-d');  
    
         $file = fopen("/home/acctmnt/logs/acctlog.txt", 'a') or die("can't open file");
         fwrite($file,"$username:m1ad $today:$time $username Email Account Successful \r\n");
         fclose($file);
         $error_flag = "no";
         $err_message = "";
     } // error flag
} //while loop
    ?>

    }// close while
}

Or how could I do below and read by extension in order? (I know this isn't the right code)

$filem1ad = glob("dir.m1ad");

$filesm1en = glob("dir.m1ad");

$filesm1ds = glob("dir.m1ad");

$filesm1pg = glob("dir.m1ad");

Yeah, that is pretty much what I figured. The code that I posted would work just fine for that. The part that I don't know is how much of the parse code needs to be different for each file type. One way to do it is to make a function for each operation you need to do: parsePurge(), parseEnable(), etc... Then you can just switch to those different functions like I showed:
$files = glob("dir");
foreach ($files as $file)
{
  if (is_file($file) && /[A-Za-z0-9]+\.m1[A-Za-z]+/ )
      parseFile($file);
}

parseFile($file)
{
  preg_match("/\.[^\.]+$/", $filename, $matches);
  $ext = strtolower($matches[0]);
  switch ($ext)
  {
      case ".m1ds":
        parseDisable($file);
/ read file
    $fp = fopen($file, "r");
    while ($line = fgets($fp))
    {
      $username = strtok($line, ":");
      $userpass = strtok(":");
      $user_pass_length = strlen($userpass);
      echo "User Name: $user_pass_length $userpass ";

        break;
      case ".m1en":
        parseDisable($file);
        break;
      case ".m1ad":
        parseCreate($file);
        break;
      case ".m1pg":
        parsePurge($file);
        break;
      default:
        die("error");
  }
}
If you do it this way, you don't need an array for each file type because they are all parsed differently in one pass.

wheakory · August 27, 2009

The files look like this (and they may have more than one record)

(delimiter is a ":")

(filename) (record values delimited by a colon).

3333.m1ad tempp.gmail1:temp:google1:temp1234

3333.m1pg tempp.gmail1:

3333.m1en tempp.gmail1:

3333.m1ds tempp.gmail1:

So I could do something like this to read the file below (sorry for the stupid questions, but I'm looking for the most efficient method for the production code)

$files = glob("dir");
foreach ($files as $file)
{
   preg_match("/\.[^\.]+$/", $filename, $matches);
   $ext = strtolower($matches[0]);
   switch ($ext)
   {
      case ".m1ds":
         parseDisable($file);
         break;
      case ".m1en":
         parseEnable($file);
         break;
      case ".m1ad":
         parseCreate($file);
         break;
      case ".m1pg":
         parsePurge($file);
         break;
      default:
         die("error");
   }
}

parseCreate($file) {
/ /read file
    $fp = fopen($file, "r");
    while ($line = fgets($fp))
    {
      $username = strtok($line, ":");
     $familyname = strtok(":");
     $givenname = strtok(":");
     $passwordname = strtok(":");
/ /read file
    $fp = fopen($file, "r");
    while ($line = fgets($fp))
    {
      $username = strtok($line, ":");
    $familyname = strtok(":");
    $givenname = strtok(":");
    $passwordname = strtok(":");
   
      // construct event object
      // save to server      
      try {
        $quota = "50M";
        $service->createUser($username, $familyname, $givenname, $passwordname, $passwordHashFunction=null, $quota=null);         
        $query2 = "INSERT INTO gmail_add VALUES('$username', '$familyname', '$givenname', '$quota', NOW() )";
        mysql_query( $query2 ) or die(mysql_error());  
} 
catch (Zend_Gdata_Gapps_ServiceException $e) {
    if ($e->hasError(Zend_Gdata_Gapps_Error::ENTITY_EXISTS)) {
      // If this is an ENTITY_DOES_NOT_EXIST error, return null
        $err_message = $err_message . "Already EXISTS \r\n";
        $error_flag = "yes";
    }
    elseif ($e->hasError(Zend_Gdata_Gapps_Error::USER_DELETED_RECENTLY)) {
      // If this is an ENTITY_DOES_NOT_EXIST error, return null
         $err_message = $err_message . "Email Account Deleted Recently cannot create until 5 days \r\n";
         $error_flag = "yes";
    }
    elseif ($e->hasError(Zend_Gdata_Gapps_Error::ENTITY_NAME_IS_RESERVED)) {
      // If this is an ENTITY_RESERVED error, return null
         $err_message = $err_message . "Email Account Name Reserved \r\n";
         $error_flag = "yes";
    }
    else {
      // Outherwise, just print the errors that occured and exit
      foreach ($e->getErrors() as $error) {
        echo "Error encountered: {$error->getReason()} ({$error->getErrorCode()})\n";
      } // foreach
      exit();
    } // if ENTITY EXISTS
  } // Catch
        echo "user $error_flag \n>";
     if ($error_flag == "yes") {
         // define current time  
         $time = date('H:i:s');  
         // fwrite($this->fp, "$time ($script_name) $message\n");  
      
         // define the current date (it will be appended to the log file name)  
         $today = date('Y-m-d');  
         // open log file for writing only; place the file pointer at the end of the file  
      
         $file = fopen("/home/acctmnt/logs/acctlog.txt", 'a') or die("can't open file");
         fwrite($file,"$username:m1ad $today:$time $username $err_message \r\n");
         fclose($file);
         $error_flag = "no";
         $err_message = "";
     }else{
         // define current time  
         $time = date('H:i:s');  
         // define the current date (it will be appended to the log file name)  
         $today = date('Y-m-d');  
    
         $file = fopen("/home/acctmnt/logs/acctlog.txt", 'a') or die("can't open file");
         fwrite($file,"$username:m1ad $today:$time $username Email Account Successful \r\n");
         fclose($file);
         $error_flag = "no";
         $err_message = "";
     } // error flag
} //while loop
    ?>

    }// close while
}

Or how could I do below and read by extension in order? (I know this isn't the right code)

$filem1ad = glob("dir.m1ad");

$filesm1en = glob("dir.m1ad");

$filesm1ds = glob("dir.m1ad");

$filesm1pg = glob("dir.m1ad");

Yeah, that is pretty much what I figured. The code that I posted would work just fine for that. The part that I don't know is how much of the parse code needs to be different for each file type. One way to do it is to make a function for each operation you need to do: parsePurge(), parseEnable(), etc... Then you can just switch to those different functions like I showed:
$files = glob("dir");
foreach ($files as $file)
{
  if (is_file($file) && /[A-Za-z0-9]+\.m1[A-Za-z]+/ )
      parseFile($file);
}

parseFile($file)
{
  preg_match("/\.[^\.]+$/", $filename, $matches);
  $ext = strtolower($matches[0]);
  switch ($ext)
  {
      case ".m1ds":
        parseDisable($file);
/ read file
    $fp = fopen($file, "r");
    while ($line = fgets($fp))
    {
      $username = strtok($line, ":");
      $userpass = strtok(":");
      $user_pass_length = strlen($userpass);
      echo "User Name: $user_pass_length $userpass ";

        break;
      case ".m1en":
        parseDisable($file);
        break;
      case ".m1ad":
        parseCreate($file);
        break;
      case ".m1pg":
        parsePurge($file);
        break;
      default:
        die("error");
  }
}
If you do it this way, you don't need an array for each file type because they are all parsed differently in one pass.

In your opinion what is the most efficient and effective way? Also speed/performance wise?

lemmin · August 28, 2009

I would probably do it like this:

$files = glob("dir");
foreach ($files as $file)
{
   $attributes = explode(":", file_get_contents($filename));

   preg_match("/\.[^\.]+$/", $filename, $matches);
   $ext = strtolower($matches[0]);
   switch ($ext)
   {
      case ".m1ds":
         parseDisable($attributes);
         break;
      case ".m1en":
         parseEnable($attributes);
         break;
      case ".m1ad":
         parseCreate($attributes);
         break;
      case ".m1pg":
         parsePurge($attributes);
         break;
      default:
         die("error");
   }
}

Since you are going to use the entire file (or at least a block of it), it is more efficient to get all of the content you need from it in one function instead of getting each line individually. Same goes for the tokenizing; there is less overhead when calling a single function.

Sign In

Read files in directory and put into a two Dimensional Array and read content

Recommended Posts

wheakory

Link to comment

Share on other sites

lemmin

Link to comment

Share on other sites

wheakory

Link to comment

Share on other sites

lemmin

Link to comment

Share on other sites

wheakory

Link to comment

Share on other sites

lemmin

Link to comment

Share on other sites

wheakory

Link to comment

Share on other sites

lemmin

Link to comment

Share on other sites

wheakory

Link to comment

Share on other sites

wheakory

Link to comment

Share on other sites

lemmin

Link to comment

Share on other sites

Join the conversation

Browse

Activity

Important Information