Adding a wait timer to a script that indexes large amounts of data.

[jadedknight]

I've written a PHP script that traverses through many XML files, building a specific URL to get some data from the World of Warcraft Armory. However, there are thousands upon thousands pieces of text needed to be indexed. The armory stops you from taking too much at once. So somehow I have to add a timer that makes the script wait every few seconds or so and checks that the XML from the URL is actually available to be loaded into simplexml_load_file(); Here is the script:

<?php
/**
* Created by PhpStorm.
* User: Chris
* Date: Jul 9, 2010
* Time: 9:04:24 AM
* To change this template use File | Settings | File Templates.
*/

/**
* Cool code snippet to read Blizzards XML
*/
//$xml = file_get_contents($url);
//echo $xml;

/**
* Set some basic constants
* EXAMPLE URL http://www.wowarmory.com/arena-ladder.xml? p=1 ts=2 & b=Vindication
* p = current page
* ts = team_size
* b = battlegroup
*/
ini_set("user_agent", "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1");
$url = "http://www.wowarmory.com/battlegroups.xml";
$team_size = 2;
$page_count = 1;

/** @var $armory_xml Load the battlegroup XML */
$armory_xml = simplexml_load_file($url);

/**
* Grab the battlegroup url parameters
*/
foreach ($armory_xml->battlegroups->battlegroup as $battlegroup) {
    $ladder_url[] = $battlegroup['ladderUrl'];

    /**
     * Grab the realms for each BG -- not needed at the moment
     */
    //foreach ($battlegroup->realms->realm as $realm) {
    //    $realm_name[] = $realm['name'];
    //}
}

/**
* Go into each BG and into each specified bracket (2v2, 3v3, 5v5) and grab each team over the specified rating
*/
foreach ($ladder_url as $bg_ladder_url) {

    /**
     * Load the BG XML with Simplexml and get the number of pages of teams for that bracket
     */
    $armory_xml = simplexml_load_file("http://www.wowarmory.com/arena-ladder.xml?p=$page_count&ts=$team_size&$bg_ladder_url");
    $page_count = intval($armory_xml->arenaLadderPagedResult['page']);
    $max_page = intval($armory_xml->arenaLadderPagedResult['maxPage']);

    /**
     * For every page, load the XML, get the arena team URL, if rating is below specified amount move on to the next BG
     */
    while ($page_count <= $max_page) {
            $arena_url = "http://www.wowarmory.com/arena-ladder.xml?p=$page_count&ts=$team_size&$bg_ladder_url";
            $arena_xml = simplexml_load_file($arena_url);
            foreach ($arena_xml->arenaLadderPagedResult->arenaTeams->arenaTeam as $arena_team) {
                if (intval($arena_team['rating']) < 2600) {
                    /*
                     * Reset the page count for the next BG
                     */
                    $page_count = 1;
                    break 2;
                } else {
                    // EXAMPLE URL for TEAM XML
                    // http://www.wowarmory.com/team-info.xml? b=Bloodlust& r=Tichondrius& ts=2& t=We+have+ms
                    $team = $arena_team['teamUrl'];
                    $team_url[] = "http://www.wowarmory.com/team-info.xml?$bg_ladder_url&$team";
                }
            }
            $page_count++;
    }
}

foreach ($team_url as $arena_team_xml) {
    $arena_xml = simplexml_load_file($arena_team_xml);
    foreach ($arena_xml->teamInfo->arenaTeam->members->character as $player) {
        $player_list[] = $player['charUrl'];
    }
}

//EXAMPLE Stats page URL http://www.wowarmory.com/character-statistics.xml? r=Daggerspine&cn=Exploitz

foreach($player_list as $player_name) {
    $arena_url = "http://www.wowarmory.com/character-statistics.xml?$player_name&c=21";
    $arena_xml = simplexml_load_file($arena_url);
    $arena_xml->category->statistic[18]['quantity'] . " " . $arena_xml->category->statistic[19]['quantity'] . " ". $arena_xml->category->statistic[20]['quantity'] . "<br />";
}

?>

 

Eventually I will add statements to save each section to a database and the script will only run every couple of days. Before I implement the database saves, how would I approach this timer and validation issue?

 

[jskywalker]

timer issue is simple:

http://nl3.php.net/manual/en/function.sleep.php

 

for the validation issue:

i'm not playing Wow, because EVE-Online is much cooler......