in_array always returning false, when it should be true?

[cyberkiller]

I am trying to take all the information from $results and input it into a text file, sometimes we get duplicates in $results so I want to check the text file for the entry before writing it in so we don't get duplicates in the text file. However inarray keeps returning false, even when I echo out the two arrays and they match?

 

$myfile = "complete.txt";

foreach ($results['info'] as $data)
{

  	$output=$fields['firstname']."|".$record['lastname']."|".$record['address']."|".$record['number']."\n";
				   
				   
$handle = fopen($myfile, "r");
$contents = fread($handle, filesize($myfile));
fclose($handle);

$list = array();

$list = explode("\n",$contents);
$list = array_map("trim", $list);

$current = $output;


echo in_array($current,$list) ? $current.' exists' : $current.' does not exist';

	if (in_array($current,$list))
	{
	print "duplicate";
	}
	else
	{

                        if($file=fopen($myfile, "a")) {  //open file for writing
                    fwrite($file, $output);      //write to file
					}
                        
					}
      
}

[johnny86]

That is because you have \n in the end of $current and you use trim to get rid of it for $list. They will not match because of the linebreake in $current.

 

Try:

echo in_array(trim($current),$list) ? $current.' exists' : $current.' does not exist';

[cyberkiller]

ok, that definitely helped. However I am still getting duplicates in the text file. It seems to allow one duplicate before it starts detecting them. There must be an error in my loops somewhere?

 

My updated code,

$myfile = "complete.txt";

foreach ($results['info'] as $data)
{

     $output=$fields['firstname']."|".$record['lastname']."|".$record['address']."|".$record['number']."\n";
                  
                  
   $handle = fopen($myfile, "r");
   $contents = fread($handle, filesize($myfile));
   fclose($handle);

   $list = array();

   $list = explode("\n",$contents);
   $list = array_map("trim", $list);

   $current = $output;

   $current = trim($current);


   echo in_array($current,$list) ? $current.' exists' : $current.' does not exist';

      if (in_array($current,$list))
      {
      print "duplicate";
      }
      else
      {
                  
                        if($file=fopen($myfile, "a")) {  //open file for writing
                       fwrite($file, $output);      //write to file
                  }
                       
                  }
     
}

[johnny86]

What exactly are you looping trough? What are these variables like $record, $results, $field and in foreach you define each line as $data and you never use that variable?

 

Also you have useless variables inside the loop like $current which equals to $output without the \n. At every iteration you open your file handle again, that is not efficent. Open it before your loop and close it after.

 

But the main point was, where do all these variables come from? You should simplify your code. For example:

<?php

// Datafile
$myFile = "myfile.txt";

// Get each line from a file to an array. Lines fetched without the ending \n or \r\n. And empty lines are skipped.
$dataArray = file($myFile, FILE_IGNORE_NEW_LINES | FILE_SKIP_EMPTY_LINES);

// Loop trough your results
foreach($results as $row)
{
    // Make the desired row from $row
    $output = $row['firstname']."|".$row['lastname']."|".$row['address']."|".$row['number'];
    
    // Check if that row already exists in the file
    if(!in_array($output, $dataArray))
    {
        // Doesn't exist, write the line and add a line break at the end.
        // FILE_APPEND will append the new line, so no need to write everything again.
        file_put_contents($myFile, $output . '\n', FILE_APPEND);
    }
}

?>

This was just an example offcourse and would not work for you if copy pasted. I just wanted to give you the idea.

[cyberkiller]

Sorry the code I posted had some typo's in it, here is the fixed version.

 

$results['info'] is the result of an API call to the back end to retrieve the data. And then firstname, lastname, address, number are the values in the array. I added the | in $output to pipe delimit the file. And basically the API data is full of duplicates, so I put in the inarray so we can clean out the duplicates as we write the text file. The whole purpose of this script is to grab data from the API and put it into a text file that can be used in excel, etc.

 

Dumping the API data was the simple part, but I was hoping to remove the duplicates as we dumped it, that's why it is reading the file every time inside the loop to keep comparing the data we just inserted to what we're about to write again. The other problem is the API has result limits, so we have to call it multiple times to get all the data out. I tried converting my code to your example, however I was having problems with $dataArray and again inarray always being false.

 

The code below works, however it doesn't seem to catch duplicates until their is already one. If I feed the exact same data into it, say 6 times. It'll insert it into the text file twice, then start reading the duplicates and skipping them. I need to get it so it's only inserting the same data in once and not twice.

 

$myfile = "complete.txt";

foreach ($results['info'] as $fields)
{

     $output=$fields['firstname']."|".$fields['lastname']."|".$fields['address']."|".$fields['number']."\n";
                 
                 
   $handle = fopen($myfile, "r");
   $contents = fread($handle, filesize($myfile));
   fclose($handle);

   $list = array();

   $list = explode("\n",$contents);
   $list = array_map("trim", $list);

   $current = $output;

   $current = trim($current);




      if (in_array($current,$list))
      {
      
      }
      else
      {
                 
                        if($file=fopen($myfile, "a")) {  //open file for writing
                       fwrite($file, $output);      //write to file
                  }
                       
                  }
     
}

[cyberkiller]

Updated the code with your example, but I am still getting duplicate entries. If I feed the script the same duplicate data say, 6 times. It will write it twice to the file, before recognizing it as being duplicate.

 

$myFile = "complete.txt";
  
  foreach($results['info'] as $fields)
{
    // Make the desired row from $row
    $output = $fields['keyword']."|".$fields['affiliate_url']."|".$fields['image_url']."|".$fields['description'];

$dataArray = file($myFile, FILE_IGNORE_NEW_LINES | FILE_SKIP_EMPTY_LINES);
    
    // Check if that row already exists in the file
    if(!in_array($output, $dataArray))
    {
        // Doesn't exist, write the line and add a line break at the end.
        // FILE_APPEND will append the new line, so no need to write everything again.
        file_put_contents($myFile, $output."\n", FILE_APPEND);
    }
}

[johnny86]

Well this takes the current data out of the file, loops trough your results and adds every line that isn't in the current data to $append array. After the loop it puts all the contents that are supposed to be appended to your file:

<?php
$myFile = "complete.txt";
$dataArray = file($myFile, FILE_IGNORE_NEW_LINES | FILE_SKIP_EMPTY_LINES);
$append = array();
foreach($results['info'] as $fields)
{
    // Make the desired row from $row
    $output = $fields['keyword']."|".$fields['affiliate_url']."|".$fields['image_url']."|".$fields['description'];
   
    // Check if that row already exists in the file
    if(!in_array($output, $dataArray))
    {
        $append[] = $output;
    }
}
file_put_contents($myFile, '\n' . implode('\n', $append), FILE_APPEND);
?>

 

Maybe that could work?

[johnny86]

Okay I just think I figured it out. So if the data file is empty you will be able to insert duplicate lines, so what you need to do is select only the unique values of that array that is being appended:

<?php
$myFile = "complete.txt";
$dataArray = file($myFile, FILE_IGNORE_NEW_LINES | FILE_SKIP_EMPTY_LINES);
$append = array();
foreach($results['info'] as $fields)
{
    // Make the desired row from $row
    $output = $fields['keyword']."|".$fields['affiliate_url']."|".$fields['image_url']."|".$fields['description'];
   
    // Check if that row already exists in the file
    if(!in_array($output, $dataArray))
    {
        $append[] = $output;
    }
}
file_put_contents($myFile, '\n' . implode('\n', array_unique($append)), FILE_APPEND);
?>

[cyberkiller]

Tried both your examples, same exact problem with both still. Two duplicates then none.

 

I just don't understand why the second one gets through, file is empty so it inserts it, second time though it should read the file and flag it as duplicate?

[PFMaBiSmAd]

You should remove duplicates in $results['info'] BEFORE you start processing the data - array_unique

 

Where is $results['info'] coming from, because there may be a better way of doing this?

 

[cyberkiller]

$results['info'] is loaded from a fetch to an API. The problem with processing it before hand is the API we are dealing with only allows 10 results at a time, so we loop it several hundred times to get the data out.

 

I'm thinking I should either just load it all into a mysql table instead of a text file and work with it there or just load the text file with the duplicates and filter them out after.

[PFMaBiSmAd]

Actually, since your data is an array of arrays, AFAIK array_unique() won't remove duplicates in each set of 10.

 

You are likely getting duplicates because some of the various fields in each set of data probably contain new-line or other white-space/non-printing characters. You should probably use an editor that has the ability to display new-line/white-space/non-printing characters to examine the data being written to the file so that you can find out why the code is storing what appear to be duplicates.

 

You do realize that your code will be simpler and much much faster if you use a database instead of a file.