Duplicate Items

[Schlo_50]

Hi there,

 

I have a rather long .DAT file that contains item details. Over time there has become duplicate items in the list and now I'd like to be able to display all dupes on a page. (Not automatically delete) I've tried to think of a way to find all repeated items but cannot get my head around it. The only thing the duplicate items have in common is that they have identical $reference variables.

 

File structure is this:

 

$Id|$Name|$Category|$Reference|$Price

 

Could someone please help me please?

Thanks in advance!

[DoddsAntS]

Hi,

 

Here is one way

 

1. Read file

2. create an array for ids and for duped ids

3. Loop through ids

    3a. Check to see if id is in array of id's if not add it if it is check to see if its in the duped id array if not add it

 

4. Echo out the ids from the duped id array

 

Hope this helps a little

[samshel]

<?php 

$arrRecords = file("/filepath/filename.dat");
$intCount = count($arrRecords);
$arrReferences = array();
$arrReferenceRecords = array();
for($i=0;$i<$intCount;$i++) {
  $arrColumns = explode("|", $arrRecords[$i]);
  if(isset($arrReferences[$arrColumns[3]])) $arrReferences[$arrColumns[3]] = $arrReferences[$arrColumns[3]] + 1;
  else $arrReferences[$arrColumns[3]] = 1;  
  
  $arrReferenceRecords[$arrColumns[3]] = $arrRecords[$i];
}
echo "Duplicate Rows...</br>";
foreach($arrReferences as $key=>$value){
  if($value > 1) {
     echo $arrReferenceRecords[$key]."</br>";
  }
}
?>

 

PS: Code is not tested...

[Schlo_50]

Thats awesome.

 

Can that code be manipulated to display both of the duplicates? At present it will just display one of the duplicates, but it would be cool to see both next to each other for the sake of comparing values.

 

Thanks

[samshel]

<?php 

$arrRecords = file("/filepath/filename.dat");
$intCount = count($arrRecords);
$arrReferences = array();
$arrReferenceRecords = array();
for($i=0;$i<$intCount;$i++) {
   $arrColumns = explode("|", $arrRecords[$i]);
   if(isset($arrReferences[$arrColumns[3]])) $arrReferences[$arrColumns[3]] = $arrReferences[$arrColumns[3]] + 1;
   else $arrReferences[$arrColumns[3]] = 1;  
   if(!isset($arrReferenceRecords[$arrColumns[3]])) $arrReferenceRecords[$arrColumns[3]] = array();
   $arrReferenceRecords[$arrColumns[3]][] = $arrRecords[$i];
}
echo "Duplicate Rows...</br>";
foreach($arrReferences as $key=>$value){
   if($value > 1) {
      echo implode(" -- ", $arrReferenceRecords[$key])."</br>";
   }
}
?>

 

 

something like this perhaps...code not tested again

[Barand]

show dupe refs and the ids of their records

<?php
$fp = fopen('datafile.dat');
$data= array();
while ($row = fgetcsv($fp,1024,'|'))
{
    list($id,,,$ref,) = $row;
    $data[$id] = $ref;
}
fclose($fp);

$counts = array_count_values($data);
foreach ($counts as $ref=>$n)
{
if ($n>1) 
{
	echo "Dupe ref : $ref in record ids " . join (', ', array_keys($data, $ref)) . '<br>';
}
}
?>

[Cleaner007]

Does anybody know how to remove duplicate files? What tool would you recommend for this? I've heard Clone Remover is the best one. How d'you think?