laatste break uit een file verwijderen

[metamlmini]

Goedenmiddag allen!

 

Ik ben bezig met een php cronscript om een text file in te lezen in een database. Die text file krijgen we van een leverancier en daardoor hebben we geen inspraak over de opmaak etc.

 

Dit was eigenlijk niet zo een probleem totdat ze ineens hadden besloten om een break in de laatste regel van de file te plaatsen... Die break krijg je wel weg met regex maar ik krijg het niet voor elkaar om dit alleen op de laatste regel van de file uit te voeren.

 

Tot nu heb ik dit:

 

$org = "org-voorraad.txt";
$f = "edit-voorraad.txt";

$string=file_get_contents("$org");
$string = str_replace("\r", "", $string);
$string = str_replace("\n", "", $string);
file_put_contents("$f", "$string");
echo "File gecheckt op breaks <br />";

 

Dit stukje code haalt alle breaks weg uit de txt file, wat dus als gevolg heeft dat de rest mijn script niet meer om weet te gaan met de data.

 

Even een stukje uit de txt file voor de beeldvorming:

 

045073	NO	629.95	 
045074	NO	629.95	 
045075	NO	629.95	 

 

Hoe haal ik in godsnaam de break weg uit de laatste regel van het txt bestandje?

 

Alvast bedankt voor de replys :-)

[metamlmini]

A damned i posted in Dutch... Sorry guys i thought that i was on a dutch forum. DOH!

 

Anyway i will translate:

 

I am working on a php cronscript that reads a txt file and places changes in a database. We recieve this txt file from another company and they will not change the way they send us this file.

 

This was never an issue until yesterday. The last line in the txt file is a break, not a html break. So now i am struggeling to remove the last break from this file.

 

In the script above you can see that i can remove all the breaks in the file but this messes up the whole script so i just want to remove the break from the lats line of the file.

 

I posted a piece of tekst from the txt file to show you how that looks.

 

I hope that you can help me out.

 

Thanks in advance!

 

PS. Is it just me or is the edit post button gone? I wanted to edit my horrible mistake in stead of replying to myself and feeling like an ass...

[Pikachu2000]

Since you're reading the entire file into a string, you can just rtrim it to remove the trailing break.

 

$string=file_get_contents("$org");
$string = rtrim($string);
file_put_contents("$f", "$string");
echo "File gecheckt op breaks <br />";

[Maq]

PS. Is it just me or is the edit post button gone? I wanted to edit my horrible mistake in stead of replying to myself and feeling like an ass...

It's not you.  We don't allow users to modify their posts after a certain amount of time, not sure exactly how long though.

[metamlmini]

Thanks for the replys

 

@Pikachu2000 Well this works fine, but this will remove all the returns from the file. Which causes some major problems in the following array. So how can i trim the return from the last line in stead of everything in the file?

 

@Maq Hmm ok, so this horrible post will last a while longer, sorry bout that :-(

[Maq]

@Maq Hmm ok, so this horrible post will last a while longer, sorry bout that :-(

No problem.  Maybe our Dutch user base will increase

[Psycho]

...Well this works fine, but this will remove all the returns from the file. Which causes some major problems in the following array. So how can i trim the return from the last line in stead of everything in the file?

 

Did you try it? rtrim() will only trim whitespace from the END of the string. So, it would not be trimming anything before the last non white-space character. Inthe example data you posted above it would trim everything after the last number 5.

 

EDIT: Personally, when processing a file such as that I prefer to use file() to read the file as an array with each line a different item in the array. Then I process each line using appropriate validation. In the case above I would process each line and "break" it into the three variables listed. If the line did not contain all three variables I would skip it (and provide appropriate error handling as needed). So, if there was an extra line break at the end (or many) they would be skipped in the processing.

[Pikachu2000]

Thanks for the replys

 

@Pikachu2000 Well this works fine, but this will remove all the returns from the file. Which causes some major problems in the following array. So how can i trim the return from the last line in stead of everything in the file?

 

 

Did you even try it?

[Psycho]

Thanks for the replys

 

@Pikachu2000 Well this works fine, but this will remove all the returns from the file. Which causes some major problems in the following array. So how can i trim the return from the last line in stead of everything in the file?

 

 

Did you even try it?

 

I'm thinking he may have forgotten to take out the previous code that removes all the line breaks

[Pikachu2000]

Thanks for the replys

 

@Pikachu2000 Well this works fine, but this will remove all the returns from the file. Which causes some major problems in the following array. So how can i trim the return from the last line in stead of everything in the file?

 

 

Did you even try it?

 

I'm thinking he may have forgotten to take out the previous code that removes all the line breaks

 

I suppose that's possible, but I did post the correct code up there . . .

[metamlmini]

HA! It worked after all, sorry i am staring at this script for a while now... 

 

$string=file_get_contents("$org");
$string = rtrim($string,"\n");
file_put_contents("$f", "$string");
echo "File gecheckt op breaks <br />";

 

The "next line" was the problem. So this is working now :-) Thanks guys!!

 

@mjdamato i was thinking to build such a error handling in this script but i dont know how. Well a little... I understand the theory.

 

Later on in this script a create a array:

 

while (!feof($file_handle) ) {
	$line_of_text = fgets($file_handle);
	$parts = explode("\t", $line_of_text);
	// Debugging
	//echo "File is uitgelezen en een array geworden! <br />";
	//print $parts[0]. $parts[1]. $parts[2]. "<BR>";

 

I should check if $part[0] is not empty else drop entry...

 

I dont know how to code that bit... A boost would help a whole lot :-)

 

[AbraCadaver]

Depending on what else you're doing, you can replace all of your code with:

 

$lines = file($org);
foreach($lines as $line) {
   $parts = explode("\t", $line);
   print_r($parts);
}

 

[Psycho]

Well, if you implement appropriate parsing/error handling in the processing logic you don't need to worry about trimming the line break at the end.

 

As I stated above I would personally use file() instead of fgets(). Also, it looks like in your original code you are creating a copy of the input file and then processing that file. Is there a reason for that? Seems unnecessary. I would do something like this:

 

//Define input file
$org = "org-voorraad.txt";

//Read input file into an array
$data = file($org);

//Process the data
foreach($data as $line)
{
    //Parse line
    $partRecord = explode("\t", $line);

    //Check that it is a valid record
    if(count($partRecord) != 3)
    {
        //Not a valid record, do not process
        //Add error handling as needed
  
        //Proceed to next record
        continue;
    }
    //Record appears valid, continue with processing

    //Remove any leading/trailing white-space in values
    array_map('trim', $partRecord);
    //Populate data array into descriptive variables
    // to make processing code more intuitive
    list($partNumber, $inStock, $price) = $partRecord;

    //Continue with processing script
    //...
    //...
    }
}

[metamlmini]

Ok, well that seems to be a great way to check if the array is populated. Thanks!

 

Well this  topic is going way deep in my script so im going to ask another thing :-)

 

The array is going to be parsed into two mysql querys.

1.

 $q = "SELECT `barcode`, `voorraad` from ".$db.".products WHERE voorraad>".$voorraad." AND barcode=".$parts[0].""; 

 

2.

 $q = "SELECT `barcode`, `voorraad` from ".$db.".products WHERE voorraad<".$voorraad." AND barcode=".$parts[0].""; 

 

But this takes a while before everything is executed. Because the same query is executed over and over again. So its not really efficient... Do you guys have any tips on this?

 

thanks guys

 

 

[Psycho]

You shuld never run queres in loops. 99% of the time there is a way you can achieve the same results by running one query. The other 1% of the time you are just plain trying to do somethign you shouldn't be.

 

Just looking at those two queries, I don't see why you need both. Why not a single query looking for all records greater than and less than the specified value. But, as I said above you would not want to be runnign queries against each value in the import file.

 

Please explain what you are trying to accomplish and I'll try and provide some code or direction.

[metamlmini]

Ok i will try to explain ;-)

 

As i stated earlier there is a file that we recieve from a supplier. I read this file using above code and create a array. The variables from that array are put into the query that i state above.

 

The select statement is a test run for an update statement. Because i want the database to be synchronised with the file that we recieve from the supplier.

 

So if the number in the file is greater or less then the database do a update of the row "voorraad" (which means stock)

 

In the mean time i replaced the select query with a update query

 

"UPDATE ".$db.".products SET voorraad=".$voorraad." WHERE voorraad < ".$voorraad." AND barcode=".$parts[0]."";

 

But this statement takes way to long.

 

I hope that this is clear enough to work with?

[Psycho]

As I said, running queries in loops is really bad - it sucks up server resources like crazy.

 

So if the number in the file is greater or less then the database do a update of the row "voorraad" (which means stock)

Think about that for a minute. You are apparently running two queries to see if the value is less than or greater than the current value. Then, after doing that comparison in PHP, you are apparently running a third query to update the value. Why not just update the value no matter what? If the value is the same - so what - it will still be the same after the update. That alone would replace three queries with just one! Plus, you don't have to add additional PHP logic.

$q = "UPDATE {$db}.products
      SET voorraad = {$voorraad}
      WHERE barcode = {$parts[0]}";

 

But, that still doesn't solve your problem of running queries in a loop. Trying to update multiple records with different values in one query is a little tricky but is possible. I'll have to research a minute and I'll post back with some sample code.

[Psycho]

OK, here is an article on how to create a single UPDATE query for multiple records needing differnt update values. http://www.karlrixon.co.uk/articles/sql/update-multiple-rows-with-different-values-and-a-single-sql-query/

 

So, in your loop you will want to create the data for the query. Then after the looping of the data is complete run the one single query to update all the records in one shot. Your performance should drastically improve.

 

Here is a modification of the previous code I posted with example logic that would allow you to run one single update query for all records

/Define input file
$org = "org-voorraad.txt";

//Read input file into an array
$data = file($org);

//Create temp arrays for query
$barCodes = array();
$whenClauses = array();
//Process the data
foreach($data as $line)
{
    //Parse line
    $partRecord = explode("\t", $line);

    //Check that it is a valid record
    if(count($partRecord) != 3)
    {
        //Not a valid record, do not process
        //Add error handling as needed
  
        //Proceed to next record
        continue;
    }
    
    //Record appears valid, continue with processing

    //Remove any leading/trailing white-space in values
    array_map('trim', $partRecord);
    //Populate data array into descriptive variables
    // to make processing code more intuitive
    list($barcode, $inStock, $qty) = $partRecord;

    //Add data to temp arrays
    $barCodes[] = $barcode;
    $whenClauses[] = "  WHEN {$barcode} THEN {$qty}";
}

//Create the single query to update ALL records
$query = "UPDATE {$db}.products
            SET voorraad = CASE barcode"
              . implode('', $whenClauses) . "
            END
          WHERE barcode IN (" . implode(',', $barCodes) . ")";

[metamlmini]

holy cr*p that is some food for thought :-) Thanks loads for explaining all this. I really want to understand all this so im not using it right away so dont think your doing it all for nothing :-).

 

Well up next is a complaint of the client that i am not updating the price. So now im strugling because the client wants to calculate 10% off haha.

 

So i never did that before but i tried:

 

	$prijs1 = &parts[2] / 10; <- line 62
	$prijs2 = $prijs1 * 100;
	$prijs = number_format(prijs2, 2, ',', '');
	// debugging
	echo $prijs. "<br />";

 

But this show the following error:

 

Parse error: syntax error, unexpected '[', expecting T_PAAMAYIM_NEKUDOTAYIM in /home/domains/public_html/splash-voorraad.php on line 62

 

I dont understand, how is it possible that i cant create a variable this way?

[Psycho]

Shouldn't that be a dollar sign and not an ampersand???

 

$prijs1 = &parts[2] / 10; <- line 62

 

I really want to understand all this so im not using it right away so dont think your doing it all for nothing :-)

Have you at least tried it? As I said the performance will increase greatly. Not, 10%, 20%, or 50%, but more like 1,000 - probably more, especially if you have many records.

[metamlmini]

I'm sorry i was kinda stressed out yesterday, the client was getting impatient so i wanted it up and running before i got to testing your script.

 

So thanks for the stupid error i created, didnt even notice that until you said it...

 

Now i have the time to check out your script.

Alright i did some edits to check some stuff like i did in my old script and ran it.

 

No errors occured, but the script doesnt create a array out of the file. I checked out the php.net on 'file' but the only reason i can think of is that i define a whole path in stead of only a filename... but that would be kinda crazy right?

 

So whats the deal, no error but no output?

 

This is the output of the file:

Variablen gezet!

Mysql connectie is een succes!

Array

UPDATE motorkledi_db.products SET voorraad = CASE barcode END WHERE barcode IN ()

 

This is the script (with edits for security reasons of course):

<?php
//Define input file / variabelen vastleggen
$org = "/full/path/to/the/file/public_html/splash/sd-voorraad-edit.txt";
$host = 'localhost';
$user = 'username';
$pass = 'password';
$db = 'dbname';

// Debugging
echo "Variablen gezet! <br />";

// Connectie met mysql
$dbconnect = mysql_connect($host, $user, $pass);
	if (!$dbconnect) {
	die('Could not connect: ' . mysql_error());
	}
// Debugging
	echo 'Mysql connectie is een succes! <br />';

//Read input file into an array
$data = file($org);

echo $data. "<br />";

//Create temp arrays for query
$barCodes = array();
$whenClauses = array();
//Process the data
foreach($data as $line) {
    //Parse line
    $partRecord = explode("\t", $line);

    //Check that it is a valid record
    if(count($partRecord) != 3){
        //Not a valid record, do not process
        //Add error handling as needed
  
        //Proceed to next record
        continue;
    }
    
    //Record appears valid, continue with processing

    //Remove any leading/trailing white-space in values
    array_map('trim', $partRecord);
    //Populate data array into descriptive variables
    // to make processing code more intuitive
    list($barcode, $qty, $price) = $partRecord;

    //Add data to temp arrays
    $barCodes[] = $barcode;
    $whenClauses[] = "  WHEN {$barcode} THEN {$qty}";
}

//Create the single query to update ALL records
$query = "UPDATE {$db}.products
            SET voorraad = CASE barcode"
              . implode('', $whenClauses) . "
            END
          WHERE barcode IN (" . implode(',', $barCodes) . ")";
	  
echo $query;
	  
?>

[Psycho]

Ok, the root of the problem is that the $whenClauses array doesn't have any values. So, just start at the beginning and work through each process that could affect those values.

 

Step 1: Reading the file

when you read the file, with file(), and echo the results with

//Read input file into an array
$data = file($org);
echo $data. "<br />";

 

You are getting

Array

 

If it wasn't reading the file the function file() will return false. So, the fact that $data is an array means that it is succesfully reading the file. Instead of echo, you should use print_r() to check the contents of the array. So, since the file is being read into an array it could be that it is an empty array, but assuming it is not then we need to check the next step.

 

Step 2: Validating the lines (records)

So, looking further on, there is the block of code to reject any lines (i.e. records) that do not consist of three values after being exploded. It could be that there is an extra tab character or something else that is causing all the lines to be rejected. You should add some debugging code to check that.

 

Step 3:

Assuming the validation is not rejecting valid records the next block of code is used to trim the values and add the values to the arrays $barCodes and $whenClauses. Since those array elements will get created even if the record values have empty strings AND $whenClauses has some static text we can be certain that these lines are not being run - otherwise the query created with those arrays would have some of that data. The only reason the query would not have any of that data is if there was a typo between the variable names - and that is not the case. So, I have to assume that either the file is not being read correctly OR the validation is rejecting the records.

 

Try running the following code with some additional debugging added to help determine where the problem is.

<?php

//Create DEBUG switch and function
$debug = true;
function debug($msg)
{
    global $debug;
    if(!$debug) { return false; }

    echo "<b>Debug</b>: {$msg}<br />\n";
    return;
}

//Define input file / variabelen vastleggen
$org = "/full/path/to/the/file/public_html/splash/sd-voorraad-edit.txt";

// Connectie met mysql
$host = 'localhost';
$user = 'username';
$pass = 'password';
$db   = 'dbname';
$dbconnect = mysql_connect($host, $user, $pass);

if (!$dbconnect)
{
    die('Could not connect: ' . mysql_error());
}

//Read input file into an array
$data = file($org);

//Add debug message to check file read
if(!$data)
{
    debug("Unable to read data file {$org}");
}
else
{
    $numberOfLines = count($data);
    debug("Data file read succesfully with {$numberOfLines} lines.", );
}
  
//Create temp arrays for query
$barCodes = array();
$whenClauses = array();
//Process the data
foreach($data as $lineNo => $line) {
    //Parse line
    $partRecord = explode("\t", $line);

    //Check that it is a valid record
    if(count($partRecord) != 3)
    {
        debug("Line $lineNo has ".count($partRecord)." elements. NOT VALID<br />\n'{$line}'");
        //Not a valid record, do not process
        //Proceed to next record
        continue;
    }
    else
    {
        debug("Line $lineNo has ".count($partRecord)." elements. VALID<br />\n'{$line}'");
    }
    
    //Record appears valid, continue with processing

    //Remove any leading/trailing white-space in values
    array_map('trim', $partRecord);
    //Populate data array into descriptive variables
    // to make processing code more intuitive
    list($barcode, $qty, $price) = $partRecord;

    //Add data to temp arrays
    $barCodes[] = $barcode;
    $whenClauses[] = "  WHEN {$barcode} THEN {$qty}";
}

//Create the single query to update ALL records
$query = "UPDATE {$db}.products
            SET voorraad = CASE barcode
            " . implode('', $whenClauses) . "
            END
          WHERE barcode IN (" . implode(',', $barCodes) . ")";
echo $query;

?>