checking for duplicates when writing to text file

[cobusbo]

Hi Im trying to write info to a text file and it works perfectly with the following script, but it writes duplicate info, how can I change it so that no duplicate records are being added to my file and just ignore inserting it without giving a message?

 

<?php
$myfile = fopen("users.csv", "a+") or die("Unable to open file!");
$mxituid = $_SERVER["HTTP_X_MXIT_USERID_R"];
$txt = "$mxituid\n";
fwrite($myfile, $txt);
fclose($myfile);
?>

Edited September 27, 2014 by cobusbo

[Frank_b]

Only possible if you first read the users.csv into memory and then scan every entry to see if there is a match. You should better store this into a database.

[cobusbo]

Only possible if you first read the users.csv into memory and then scan every entry to see if there is a match. You should better store this into a database.

It doesn't have any private info in it the reason I want to store it as this file type is because I need to link the file to another website to publish something to all users saved in this file. So I don't want duplicate records

[Frank_b]

You can 'link' data from the database to other websites in various ways.

 

The point is that databases gives you a normalized way to store huge amounts of records and a fast way to search those records.

 

Its much more faster then using a csv file.

[Barand]

Perhaps a combination of

• file(), to read the csv file into an array and
• array_unique(), to remove any duplicates

[cobusbo]

 

Perhaps a combination of

• file(), to read the csv file into an array and
• array_unique(), to remove any duplicates

 

Thank you It solved my problem

 

 

<?php
$myfile = fopen("users.csv", "a+") or die("Unable to open file!");
$mxituid = $_SERVER["HTTP_X_MXIT_USERID_R"];
$txt = "$mxituid\n";
fwrite($myfile, $txt);
fclose($myfile);




$list = file('users.csv');
$list = array_unique($list);
file_put_contents('unique.csv', implode('', $list));


?>

[Ch0cu3r]

There is no need to write an entry to users.csv and then read the contents of the file filtering out the duplicates and then writing the contents back to the file.

 

Instead. Read the contents of the file and and then only write the value in $_SERVER["HTTP_X_MXIT_USERID_R"] to the file if it does not exists

$file = 'users.csv';
$mxituid = $_SERVER["HTTP_X_MXIT_USERID_R"];

// read contents of file
$list = file($file, FILE_IGNORE_NEW_LINES);

// if the value is not logged
if(!in_array($mxituid, $list))
{
   // append the value to the file
   file_put_contents($file, $mxituid . PHP_EOL, FILE_APPEND);
}