[SOLVED] Help RETRIEVING results from alltheweb!!please

Trenty · December 20, 2007

Hi ...

This is my first post in this forum & and am hoping that you can help me with this code:

this is a class witch request ( grabbing audio search results from Alltheweb.com ) this is the code

<?
class AudioSearch
{
  var $m_url = 'search?cat=mp3&cs=utf8&q={keyword}&rys=0&itag=crv&_sb_lang=any';
  var $m_host = "www.alltheweb.com";
  //var $m_url = '/demo/allweb/?search={keyword}&type=audio';
  //var $m_host = "www.script66.com";
  var $m_settings;
  var $m_total_found_string;
  var $m_total_found_int;
  var $m_keywords;
  var $m_pager;
  var $m_is_no_resuls = 0;

  function AudioSearch($settings,$keywords)
  {
    $this->m_settings = $settings;
    $this->m_keywords = $keywords;
  }
  function file_get_contents($url)
  {
    if (check_if_bad_request($this->m_keywords))
    {
      return '';
    }
    $out = "GET /$url HTTP/1.1\r\n";
    $out .= "Host: {$this->m_host}\r\n";
    $out .= "User-Agent: Mozilla 4.0\r\n";
    $out .= "Connection: close\r\n\r\n";
    
    $h = fsockopen($this->m_host,80);
    fwrite($h,$out);
    for($a = 0,$r = '';!$a;)
    {
       $b = fread($h,8192);
       $r .= $b;
       $a = (($b=='') ? 1 : 0);
    }
    fclose($h);
    return $r;
  }
  function get_url()
  {
    if (_DEBUG == 1)
      return 'www/audio.htm';
    else
    {
      $this->m_url = str_replace('{keyword}',$this->m_keywords,$this->m_url);
      $curr_page = isset($_GET['pos']) ? $_GET['pos'] : 1;
      if ($curr_page != 1)
      {
         $curr_page = $curr_page * 10 - 10;
         $this->m_url .= "&o=$curr_page";
       }
      return $this->m_url;
    }
  }
  function get_total_found_string($content)
  {
    if ($this->m_is_no_resuls == 1)
      return "";
    preg_match('|(<span class="noFound">.*<span class="ofSoMany">(.*)</span>)|Ui',$content,$ret);
    $this->m_total_found_int = (int)str_replace(',','',$ret[2]);
    $string = str_replace(' of ',' &#1605;&#1606; ',$ret[1]);
    return $string;
  }

  function get_results()
  {
    $url = $this->get_url();
    $content = $this->file_get_contents($url);
    $content = str_replace("\r\n"," ",$content);
    $content = str_replace("\n"," ",$content);
    $content = str_replace("<td class=\"colLabel\">Title</td>",'<td class="colLabel"> &#1575;&#1604;&#1605;&#1608;&#1590;&#1608;&#1593; </td>',$content);
    $content = str_replace("<td align=\"center\" class=\"colLabel\">Size</td>",'<td class="colLabel"> &#1575;&#1604;&#1581;&#1580;&#1605; </td>',$content);
    $content = str_replace("<td align=\"center\" class=\"colLabel\">Date</td>",'<td class="colLabel"> &#1575;&#1604;&#1578;&#1575;&#1585;&#1610;&#1582; </td>',$content);
    //Get total found string
    $this->m_total_found_string = $this->get_total_found_string($content);
    //Init pager
    $this->m_pager = new Pager($this->m_total_found_int,10,'audio',$this->m_keywords);

    $ret_data = '
    <div style="padding-left:20px;padding-top:20px;">
    ';
    //Get
   if (preg_match_all('|<DIV class="resultWell">.*<table.*>(.*)</div>|Ui',$content,$ret))
    {
      if (!preg_match_all('/<a.*href=".*\/\*\*(.*)".*>.*<\/a>/Ui',$ret[1][0],$links))
      {
        preg_match_all('/<a.*href="(.*)".*>.*<\/a>/Ui',$ret[1][0],$links);
      }
      $res = $ret[1][0];

      foreach($links[0] as $index=>$data)
      {
 $url = urldecode($links[1][$index]);
 //$url = $links[1][$index];
         $res = str_replace($data,"<a href=\"{$url}\" style=\"font-size:12px;\" target=\"_new\"><img src=\"images/audio2.gif\" alt=\"Go to this file\" border=\"0\" /></a>",$res);
       }
      $ret_data .=  '<table width="100%" class="mp3" cellpadding="10">'.$res;
    }
    else
     {
        $this->m_is_no_resuls = 1;
        return $this->m_settings->get_noresults_string();
     }
    $pager = $this->m_pager->get_pager();
    $ret_data .= '
    </div>
    <div style="padding-left:20px;padding-top:10px;">
    <DIV class=rnav>
	<p align="center"><SPAN class=rnavLabel>&#1575;&#1604;&#1589;&#1601;&#1581;&#1575;&#1578; : '.$pager.'</span></div>
    </div>
    ';
    return $ret_data;
  }
}
?>

The problem nothing wrong with this script( it is working fine)

But I want the Two variables :

  var $m_url = 'search?cat=mp3&cs=utf8&q={keyword}&rys=0&itag=crv&_sb_lang=any';
  var $m_host = "www.alltheweb.com";

Changes randomly from these to these

var $m_url = '/demo/allweb/?search={keyword}&type=audio';
var $m_host = "www.script66.com";

please if you can Reply ASAP

REgards

thanks In ADvance

trq · December 21, 2007

Sorry, I don't see a question.

Trenty · December 21, 2007

ok , Thank u for replying so fast

Let me rephrase

How can I Change the the following variables

  var $m_url = 'search?cat=mp3&cs=utf8&q={keyword}&rys=0&itag=crv&_sb_lang=any';
  var $m_host = "www.alltheweb.com";

With the these variables :

var $m_url = '/demo/allweb/?search={keyword}&type=audio';
var $m_host = "www.script66.com";

randomly?

thanks

trq · December 21, 2007

Replace...

var $m_url = 'search?cat=mp3&cs=utf8&q={keyword}&rys=0&itag=crv&_sb_lang=any';
  var $m_host = "www.alltheweb.com";
  //var $m_url = '/demo/allweb/?search={keyword}&type=audio';
  //var $m_host = "www.script66.com";

with...

var $m_url;
var $m_host;
var $urls = array(
  'www.alltheweb.com' => 'search?cat=mp3&cs=utf8&q={keyword}&rys=0&itag=crv&_sb_lang=any',
  'www.script66.com' => '/demo/allweb/?search={keyword}&type=audio'
);

Then place the following code within AudioSearch method.

function AudioSearch($settings,$keywords)
  {
    $n = rand(0,count($this->urls));
    $this->m_url = $this->urls[$n];
    $this->m_host = $this->urls[$n];
    $this->m_settings = $settings;
    $this->m_keywords = $keywords;
  }

You can add as many url/host pairs as you like to the array.

Trenty · December 21, 2007

thank you for your efforts...

I did what asked me to todo ... but When I executed the code The page page took some time to load then I got this Error :

Warning: fsockopen() [function.fsockopen]: unable to connect to :80 in C:\AppServ\www\includes\classes\web_search_class.php on line 37

Fatal error: Maximum execution time of 30 seconds exceeded in C:\AppServ\www\includes\classes\web_search_class.php on line 37

This is the code that I modified

<?
class AudioSearch
{
var $m_url;
var $m_host;
var $urls = array(
  'www.alltheweb.com' => 'search?cat=mp3&cs=utf8&q={keyword}&rys=0&itag=crv&_sb_lang=any',
  'www.script66.com' => '/demo/allweb/?search={keyword}&type=audio'
);
  var $m_settings;
  var $m_total_found_string;
  var $m_total_found_int;
  var $m_keywords;
  var $m_pager;
  var $m_is_no_resuls = 0;

  function AudioSearch($settings,$keywords)
  {
    $n = rand(0,count($this->urls));
    $this->m_url = $this->urls[$n];
    $this->m_host = $this->urls[$n];
    $this->m_settings = $settings;
    $this->m_keywords = $keywords;
  }
  function file_get_contents($url)
  {
    if (check_if_bad_request($this->m_keywords))
    {
      return '';
    }
    $out = "GET /$url HTTP/1.1\r\n";
    $out .= "Host: {$this->m_host}\r\n";
    $out .= "User-Agent: Mozilla 4.0\r\n";
    $out .= "Connection: close\r\n\r\n";
    
    $h = fsockopen($this->m_host,80);
    fwrite($h,$out);
    for($a = 0,$r = '';!$a;)
    {
       $b = fread($h,8192);
       $r .= $b;
       $a = (($b=='') ? 1 : 0);
    }
    fclose($h);
    return $r;
  }
  function get_url()
  {
    if (_DEBUG == 1)
      return 'www/audio.htm';
    else
    {
      $this->m_url = str_replace('{keyword}',$this->m_keywords,$this->m_url);
      $curr_page = isset($_GET['pos']) ? $_GET['pos'] : 1;
      if ($curr_page != 1)
      {
         $curr_page = $curr_page * 10 - 10;
         $this->m_url .= "&o=$curr_page";
       }
      return $this->m_url;
    }
  }
  function get_total_found_string($content)
  {
    if ($this->m_is_no_resuls == 1)
      return "";
    preg_match('|(<span class="noFound">.*<span class="ofSoMany">(.*)</span>)|Ui',$content,$ret);
    $this->m_total_found_int = (int)str_replace(',','',$ret[2]);
    $string = str_replace(' of ',' &#1605;&#1606; ',$ret[1]);
    return $string;
  }

  function get_results()
  {
    $url = $this->get_url();
    $content = $this->file_get_contents($url);
    $content = str_replace("\r\n"," ",$content);
    $content = str_replace("\n"," ",$content);
    $content = str_replace("<td class=\"colLabel\">Title</td>",'<td class="colLabel"> &#1575;&#1604;&#1605;&#1608;&#1590;&#1608;&#1593; </td>',$content);
    $content = str_replace("<td align=\"center\" class=\"colLabel\">Size</td>",'<td class="colLabel"> &#1575;&#1604;&#1581;&#1580;&#1605; </td>',$content);
    $content = str_replace("<td align=\"center\" class=\"colLabel\">Date</td>",'<td class="colLabel"> &#1575;&#1604;&#1578;&#1575;&#1585;&#1610;&#1582; </td>',$content);
    //Get total found string
    $this->m_total_found_string = $this->get_total_found_string($content);
    //Init pager
    $this->m_pager = new Pager($this->m_total_found_int,10,'audio',$this->m_keywords);

    $ret_data = '
    <div style="padding-left:20px;padding-top:20px;">
    ';
    //Get
   if (preg_match_all('|<DIV class="resultWell">.*<table.*>(.*)</div>|Ui',$content,$ret))
    {
      if (!preg_match_all('/<a.*href=".*\/\*\*(.*)".*>.*<\/a>/Ui',$ret[1][0],$links))
      {
        preg_match_all('/<a.*href="(.*)".*>.*<\/a>/Ui',$ret[1][0],$links);
      }
      $res = $ret[1][0];

      foreach($links[0] as $index=>$data)
      {
 $url = urldecode($links[1][$index]);
 //$url = $links[1][$index];
         $res = str_replace($data,"<a href=\"{$url}\" style=\"font-size:12px;\" target=\"_new\"><img src=\"images/audio2.gif\" alt=\"Go to this file\" border=\"0\" /></a>",$res);
       }
      $ret_data .=  '<table width="100%" class="mp3" cellpadding="10">'.$res;
    }
    else
     {
        $this->m_is_no_resuls = 1;
        return $this->m_settings->get_noresults_string();
     }
    $pager = $this->m_pager->get_pager();
    $ret_data .= '
    </div>
    <div style="padding-left:20px;padding-top:10px;">
    <DIV class=rnav>
	<p align="center"><SPAN class=rnavLabel>&#1575;&#1604;&#1589;&#1601;&#1581;&#1575;&#1578; : '.$pager.'</span></div>
    </div>
    ';
    return $ret_data;
  }
}
?>

Trenty · December 21, 2007

Anyone please :-[

Trenty · December 21, 2007

No REply Yet !!!!

Sign In

[SOLVED] Help RETRIEVING results from alltheweb!!please

Recommended Posts

Trenty

Link to comment

Share on other sites

trq

Link to comment

Share on other sites

Trenty

Link to comment

Share on other sites

trq

Link to comment

Share on other sites

Trenty

Link to comment

Share on other sites

Trenty

Link to comment

Share on other sites

Trenty

Link to comment

Share on other sites

Archived

Browse

Activity

Important Information