benchmarking

[.josh]

Lots of times I see threads where people like to nitpick which method of doing something is faster, and it usually boils down to benchmarking the two (or more) methods...that is, executing each method lots of times and getting the average and seeing which one on average is faster.  Here is a function I wrote that makes it easy to benchmark methods. 

 

Here is the function:

 

<?php
/**
  benchmark takes a list of user defined callback functions and
            calls each of them $rounds amount of times, recording
            the start and end times. It then calculates the average
            and then time it took to execute each function and then 
            outputs some information.  Basic principle is to find out
            which method of something is faster.  

  @param integer $rounds The number of times the user defined function
                         will be executed. This is not an optional 
                         parameter, but it defaults to 100.
  @param string  argN    The name of the user defined callback function to call. 
                         Every argument after the first argument will be used as
                         a user defined function to call. 
*/
function benchmark ($rounds=100) {
  // if an integer is not passed as first argument, default to 100
  if (!is_integer($rounds)) $rounds = 100;
  // get the rest of the arguments passed to the function
  $funcs = func_get_args();	
  // remove first argument from the list, since it is $rounds
  array_shift($funcs);
  // for each user defined function...
  foreach ($funcs as $func) {
    // if the function doesn't exist, skip it
    if (!function_exists($func)) break; 
    $time = array();
    // call the function the specified/default amount of times
    for ($c = 0; $c < $rounds; $c++) {
      // get the current microtime timestamp
      $start = explode(" ",microtime());
      $start = $start[0];

      // call the user defined function
      $func();

      // get the current microtime timestamp
      $end = explode(" ",microtime());
      $end = $end[0];
      // find out the difference between the two
      $diff = bcsub($end,$start,20);

      // for some unknown reason, $diff occasionally returns a negative number
      // dunno if it's a bug in bcsub or a bug in microtime or maybe sometimes
      // it just goes so damn fast it goes backwards in time! anyways...let's
      // just weed it out, make sure it doesn't skew the results. 
      if ($diff > 0) $time[$c] = $diff;
    } // end $c
    // get the average time
    $average[$func] = rtrim(bcdiv(array_sum($time),count($time),20),'0');
  } // end foreach funcs

  // sort the averages ascending (preserving the keys, because we used the  
  // user defined function name as the array keys
  asort($average,SORT_NUMERIC);
  // get the fastest average
  $fastest = max($average);
  // get the slowest average
  $slowest = min($average);

  // display how many times we executed each function
  echo "times executed (each): ".$rounds . "<br/>";
  // display the averages
  echo "fastest to slowest:<br/><pre>";print_r($average); echo "</pre>";
  // display the time difference between slowest and fastest
  echo "biggest difference time: " . rtrim(bcsub($fastest,$slowest,20),'0') . "<br/>";
  // calculate and display how much faster the fastest one was, compared
  // to the slowest, as a percentage
  $percent = rtrim(round(bcmul(bcsub(bcdiv($fastest,$slowest,20),1,20),100,20),4),'0');

  echo "fastest is " . $percent . "% faster than the slowest";
} // end benchmark
?>

 

Here is an example of how to use it.  In this example, I want to compare 3 different regex patterns and see which one is on average the fastest.  So I wrote a wrapper function for each one and then call the benchmark function:

 

/* example: */

$content = "[tag]lots of stuff here[/tag]";

function lazy_matchall () {
  global $content;
  preg_match("~\[tag[^\]]*\](.*?)\[/tag\]~is", $content);
}

function greedy_matchall () {
  global $content;
  preg_match("~\[tag[^\]]*\](.*)\[/tag\]~is", $content);
}

function negative_lookahead () {
  global $content;
  preg_match("~\[tag[^\]]*\]((??!\[/tag\]).)*)~is", $content);
}

benchmark(10000,'lazy_matchall','greedy_matchall','negative_lookahead');

 

Output:

 

times executed (each): 10000
fastest to slowest:

Array
(
    [greedy_matchall] => 0.0000094208
    [lazy_matchall] => 0.0000102794
    [negative_lookahead] => 0.0000116116
)

biggest difference time: 0.0000021908
fastest is 23.2549% faster than the slowest