Email Duplicate Removal More Than 20k not working

[soma56]

I've created a script to remove duplicate emails and it works perfectly up to about 20k. After that it just stops with no error code. I'm baffled. I've increased the max_execution_time to 2 hours even though this script just takes a few minutes.

 

<?PHP
function validElement($element) {
    return strlen($element) > 1;
}

function in_iarray($str, $a){
foreach($a as $v){
if(strcasecmp($str, $v)==0){return true;}
}
return false;
}

remove_duplicates();
}
//Remove more duplicates from list
function array_iunique($a){
$n = array();
foreach($a as $k=>$v){
if(!in_iarray($v, $n)){
echo $v;
$n[$k]=$v;}
}
return $n;
}

//Remove duplicates from list
function remove_duplicates() {

$p = 0;
while ($p != 1) {
   
$rawemaillist = array_values(array_filter($rawemaillist, "validElement"));
$p = 1;
}
$initial = count($rawemaillist);
$k = 0;
while ($k != 1) {
$rawemaillist = array_iunique($rawemaillist);
$k = 1;
}
?>

 

The script works fine - but only up to around 20 k - which leads me to believe its a server setting issue. Is there something in the php.ini file I should or otherwise a setting that I should be looking at to change???

[MadTechie]

its probably memory, turn error reporting on

code:

error_reporting(E_ALL);

 

why not use array_unique then filter the emails for valid emails ??

[soma56]

thank you for the response. That's exactly what i'm going to do. I'm trying to convert all the values within the array to lowercase first before using 'array_unique'.

[MadTechie]

how about

$a = array("TEST", "test2", "test3");
$b = array_map("strtolower", $a);
print_r($b);

 

[soma56]

MadTechie - you're the man - that' s what I needed to know. Thankyou.