Search the Community

Hello all, I have a video converting script fetch / remote download and then convert using FFMPEG. Recently i had issue with one of the extractor "plugin" when i try to convert i get error in the browser in Virtualmin error log i get [Wed Jul 06 20:30:12.758777 2022] [proxy_fcgi:error] [pid 842:tid 139961936357120] [client 2.xx.xxx.9:56176] AH01071: Got error 'PHP message: PHP Notice: Undefined offset: 2 in /home/testdomain/public_html/lib/extractors/Extractor.php on line 205PHP message: PHP Notice: Undefined offset: 2 in /home/testdomain/public_html/lib/extractors/Extractor.php on line 205PHP message: PHP Notice: Undefined offset: 2 in /home/testdomain/public_html/lib/extractors/Extractor.php on line 205PHP message: PHP Notice: Undefined offset: 2 in /home/testdomain/public_html/lib/extractors/Extractor.php on line 205PHP message: PHP Notice: Undefined offset: 2 in /home/testdomain/public_html/lib/extractors/Extractor.php on line 205PHP message: PHP Notice: Undefined offset: 2 in /home/testdomain/public_html/lib/extractors/Extractor.php on line 205PHP message: PHP Notice: Undefined offset: 2 in /home/testdomain/public_html/lib/extractors/Extractor.php on line 205PHP message: PHP Notice: Undefined offset: 2 in /home/testdomain/public_html/lib/extractors/Extractor.php on line 205PHP message: PHP Notice: Undefined offset: 2 in /home/testdomain/public_html/lib/extractors/Extractor.php on line 205PHP message: PHP Notice: Undefined offset: 2 in /home/testdomain/public_html/lib/extractors/Extractor.php on line 205PHP message: PHP Notice: Undefined offset: 2 in /home/testdomain/public_html/lib/extractors/Extractor.php on line 205PHP message: PHP Notice: Undefined offset: 2 in /home/testdomain/public_html/lib/extractors/Extractor.php on line 205PHP message: PHP Warning: A non-numeric value encountered in /home/testdomain/public_html/lib/extractors/yt.php on line 202PHP message: PHP Fatal error: Uncaught Error: Unsupported operand types in /home/testdomain/public_html/lib/extractors/yt.php:202\nStack trace:\n#0 /home/testdomain/public_html/lib/VideoConverter.php(258): YouTubeMp3Converter\\lib\\extractors\\yt->RetrieveVidInfo()\n#1 /home/testdomain/public_html/inc/index_header.php(175): YouTubeMp3Converter\\lib\\VideoConverter->ValidateConversionForm()\n#2 /home/testdomain/public_html/index.php(4): include('/home/testdomai...')\n#3 {main}\n thrown in /home/testdomain/public_html/lib/extractors/yt.php on line 202', referer: http://91.xxx.xxx.xxx/?config=complete [Wed Jul 06 20:36:49.899796 2022] [proxy_fcgi:error] [pid 842:tid 139961886000896] [client 2.xx.xxx.9:56487] AH01071: Got error 'PHP message: PHP Notice: Undefined offset: 1 in /home/testdomain/public_html/lib/extractors/Extractor.php on line 205PHP message: PHP Notice: Undefined offset: 1 in /home/testdomain/public_html/lib/extractors/Extractor.php on line 205PHP message: PHP Notice: Undefined offset: 1 in /home/testdomain/public_html/lib/extractors/Extractor.php on line 205PHP message: PHP Notice: Undefined offset: 1 in /home/testdomain/public_html/lib/extractors/Extractor.php on line 205PHP message: PHP Notice: Undefined offset: 1 in /home/testdomain/public_html/lib/extractors/Extractor.php on line 205PHP message: PHP Notice: Undefined offset: 1 in /home/testdomain/public_html/lib/extractors/Extractor.php on line 205PHP message: PHP Notice: Undefined offset: 1 in /home/testdomain/public_html/lib/extractors/Extractor.php on line 205PHP message: PHP Notice: Undefined offset: 1 in /home/testdomain/public_html/lib/extractors/Extractor.php on line 205PHP message: PHP Notice: Undefined offset: 1 in /home/testdomain/public_html/lib/extractors/Extractor.php on line 205PHP message: PHP Notice: Undefined offset: 1 in /home/testdomain/public_html/lib/extractors/Extractor.php on line 205PHP message: PHP Notice: Undefined offset: 1 in /home/testdomain/public_html/lib/extractors/Extractor.php on line 205PHP message: PHP Notice: Undefined offset: 1 in /home/testdomain/public_html/lib/extractors/Extractor.php on line 205PHP message: PHP Warning: A non-numeric value encountered in /home/testdomain/public_html/lib/extractors/yt.php on line 202PHP message: PHP Fatal error: Uncaught Error: Unsupported operand types in /home/testdomain/public_html/lib/extractors/yt.php:202\nStack trace:\n#0 /home/testdomain/public_html/lib/VideoConverter.php(258): YouTubeMp3Converter\\lib\\extractors\\yt->RetrieveVidInfo()\n#1 /home/testdomain/public_html/inc/index_header.php(175): YouTubeMp3Converter\\lib\\VideoConverter->ValidateConversionForm()\n#2 /home/testdomain/public_html/index.php(4): include('/home/testdomai...')\n#3 {main}\n thrown in /home/testdomain/public_html/lib/extractors/yt.php on line 202', referer: http://91.xxx.xxx.xxx/index.php Previously i got it fixed with a help of a developer and everything worked perfectly without any issue on aaPanel but when i test this in Virtualmin i get this error. I will post 4 codes and related to the extractor which is having issue below is the (01) yt.php <?php namespace YouTubeMp3Converter\lib\extractors; use \YouTubeMp3Converter\lib\Config; // hub Extractor Class class yt extends Extractor { // Fields public $_reqHeaders = array( //'Accept-Encoding: gzip, deflate', 'Accept-Language: en-us,en;q=0.5', 'Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7', 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', 'Cookie: age_verified=1; platform=pc' ); protected $_mainUserAgent = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.131 Safari/537.36'; private $_platforms = array('pc'/*, 'tv'*/); #region Public Methods function RetrieveVidInfo($vidUrl) { $converter = $this->GetConverter(); $videoInfo = array(); $vidPage = ''; $srcUrl = ''; $vidTitle = 'Unknown'; $vidImage = 'https://img.youtube.com/vi/oops/1.jpg'; $vidDuration = array(); foreach ($this->_platforms as $platform) { $vidPage = $this->FileGetContents($vidUrl, "", $this->_reqHeaders); if (!empty($vidPage)) { $reqCookies = $this->ExtractCookies(); if (empty($srcUrl) && preg_match('/<video[^>]+src="(.+?)"[^>]*><\/video>/is', $vidPage, $matches) == 1) { $srcUrl = preg_replace('/^(\/{2})/', "http://", trim($matches[1])); } if (empty($srcUrl)) { $srcUrls = array(); if (preg_match('/var player_mp4_seek = "[^"]*";\s*(\/\/[^\/]+?\n)*(.+?)\n/is', $vidPage, $matches) == 1) { //die(trim($matches[2])); if (preg_match('/var qualityItems_[a-zA-Z0-9]+\s*=\s*(\[\{.+?\}\]);/is', $matches[2], $qitems) == 1) { $items = json_decode($qitems[1], true); if (json_last_error() == JSON_ERROR_NONE) { //die(print_r($items)); $itemUrls = array(); foreach ($items as $item) { if (isset($item['url']) && !empty($item['url'])) { $srcUrls[] = $item['url']; } } arsort($srcUrls); } } if (empty($srcUrls)) { if (preg_match('/(var\s+(?:(media|quality))_.+)/', $vidPage, $assignments) == 1) { //die(print_r($assignments)); $assignmentsArr = explode(";", $assignments[0]); //die(print_r($assignmentsArr)); $media = array(); array_walk($assignmentsArr, function($val) use(&$media) { if (preg_match('/^(var\s+(media|quality(?!Items))_\d=)/', $val) == 1) $media[] = preg_replace('/\/\*[^\*]*\*\//', "", preg_replace('/^(var\s+(media|quality(?!Items))_\d=)/', "", $val)); }); //die(print_r($media)); $surl = ""; foreach ($media as $medium) { $jsKeys = preg_split('/\s*\+\s*/', $medium, -1, PREG_SPLIT_NO_EMPTY); //die(print_r($jsKeys)); foreach ($jsKeys as $jsKey) { if (preg_match('/var\s+' . preg_quote($jsKey, "/") . '=([^;]+)/', $vidPage, $jsKeyMatch) == 1) { $surl .= preg_replace('/("|\+|\s)/', "", $jsKeyMatch[1]); } } if (!empty($surl)) { //echo $surl . "<br>"; if (preg_match('/get_media/', $surl) == 1) { $this->_reqHeaders[count($this->_reqHeaders) - 1] .= ';' . $reqCookies; //die(print_r($this->_reqHeaders)); $reqTries = 0; do { $mp4Json = $this->FileGetContents($surl, "", $this->_reqHeaders); $mp4Data = json_decode($mp4Json, true); $reqTries++; } while ($reqTries < Config::_MAX_CURL_TRIES && (json_last_error() != JSON_ERROR_NONE || empty($mp4Data))); $mp4Data_pre = array() ; foreach($mp4Data as $k => $v){ if($v['quality'] > 1080) unset($mp4Data[$k]); else if($v['format'] == 'hls') { $mp4Data_pre['ff_pre'] = ' -protocol_whitelist file,tls,tcp,https,crypto -allowed_extensions ALL '; $mp4Data_pre['ff_for'] = true ; } } //die(print_r($mp4Data)); if (isset($mp4Data[count($mp4Data) - 1]['videoUrl'])) { $srcUrls[0] = $mp4Data[count($mp4Data) - 1]['videoUrl']; } } if (empty($srcUrls) && preg_match('/1080P.*?720P.*?480P.*?\.m3u8\?/', $surl) == 1) { $m3u8 = $this->FileGetContents($surl, "", $this->_reqHeaders); if (!empty($m3u8)) { $m3u8Lines = preg_split('/\n|\r/', $m3u8, -1, PREG_SPLIT_NO_EMPTY); $m3u8Lines = preg_grep('/^(#)/', $m3u8Lines, PREG_GREP_INVERT); //die(print_r($m3u8Lines)); if (!empty($m3u8Lines)) { $surl = preg_replace('/(' . preg_quote(strrchr($surl, "/"), "/") . ')$/', "", $surl); $surl = $surl . "/" . current($m3u8Lines); $srcUrls[] = $surl; } } } //if (preg_match('/\.m3u8\?/', $surl) != 1) $srcUrls[] = $surl; } $surl = ""; } //die(print_r($srcUrls)); } } if (empty($srcUrls)) { preg_match_all('/var ([^=]+)="([^"]*)"(\s*\+\s*"([^"]*)")?;/is', trim($matches[2]), $matches2); if (!empty($matches2)) { //die(print_r($matches2)); $urlParts = array(); foreach ($matches2[0] as $k => $m) { $urlParts[$matches2[1][$k]] = $matches2[2][$k] . $matches2[4][$k]; } //die(print_r($urlParts)); if (!empty($urlParts)) { preg_match_all('/var quality_(\d+)p=(.+?);/is', trim($matches[2]), $matches3); if (!empty($matches3)) { //die(print_r($matches3)); foreach ($matches3[0] as $k => $m) { $urlVars = preg_replace('/\/\*[^\*]*\*\//', "", $matches3[2][$k]); $urlVars = preg_split('/\+/', $urlVars, -1, PREG_SPLIT_NO_EMPTY); foreach ($urlVars as $uvar) { $uvar = trim($uvar); $srcUrls[$matches3[1][$k]] .= (isset($urlParts[$uvar])) ? $urlParts[$uvar] : ''; } } arsort($srcUrls); } } } } } //die(print_r($srcUrls)); $srcUrl = (!empty($srcUrls)) ? current($srcUrls) : $srcUrl; } if ($vidTitle == 'Unknown' && preg_match("/('flashvars'\s*:\s*\{(.+?)\},)|(var flashvars\w* = \{(.+?)\};)/is", $vidPage, $matched) == 1) { //die(print_r($matched)); $rawJson = (!empty($matched[2])) ? $matched[2] : $matched[4]; $json = json_decode('{' . $rawJson . '}', true); if (json_last_error() == JSON_ERROR_NONE) { //die(print_r($json)); if (!isset($json['video_title'])) { $json = json_decode('{' . $matched[4] . '}', true); } $vidTitle = (isset($json['video_title'])) ? urldecode($json['video_title']) : $vidTitle; $vidImage = (isset($json['image_url'])) ? urldecode($json['image_url']) : $vidImage; $vidDuration = (isset($json['video_duration'])) ? array('duration' => (int)$json['video_duration']) : $vidDuration; } } } } parse_str(parse_url($vidUrl, PHP_URL_QUERY), $urlVars); if (isset($urlVars['viewkey'])) { $videoInfo = array('id' => $urlVars['viewkey'], 'title' => $vidTitle, 'thumb_preview' => $vidImage, 'src_sd' => $srcUrl, 'src_hd' => $srcUrl, 'cookies' => preg_replace('/^(Cookie: )/', "", $this->_reqHeaders[count($this->_reqHeaders) - 1])) + $vidDuration + $mp4Data_pre; } //die(print_r($videoInfo)); //print_r($videoInfo); return $videoInfo; } function ExtractVidSourceUrls() { // Populate vars required for extraction $converter = $this->GetConverter(); $vidUrls = array(); $ftype = $converter->GetConvertedFileType(); $fcategory = $converter->GetConvertedFileCategory(); $vidHost = $converter->GetCurrentVidHost(); $vidInfo = $converter->GetVidInfo(); $vidHosts = $converter->GetVideoHosts(); $vidQualities = array(); array_walk($vidHosts, function($vh, $key) use(&$vidQualities, $vidHost) {if ($vh['name'] == $vidHost) $vidQualities = $vh['video_qualities'];}); // Start extraction $playUrls = array(); foreach ($vidQualities as $key => $fq) { if (!empty($vidInfo[$fq]) && !in_array($vidInfo[$fq], $playUrls)) { $vidUrls[] = array('mp4', $key, $vidInfo[$fq]); $playUrls[] = $vidInfo[$fq]; } } //die(print_r($vidUrls)); return ($fcategory == 'audio' || $ftype == '3gp') ? array_reverse($vidUrls) : $vidUrls; } #endregion } ?> (02) remote.php <?php namespace YouTubeMp3Converter\lib; use YouTubeMp3Converter\lib\extractors\Extractor; // Remote Download Class class Remote { // Private Fields private static $_converter; private static $_curlResource; private static $_percentVidDownloaded = 0; private static $_fsize; private static $_downloaded; private static $_chunkCount = 0; private static $_prevChunkCount = 0; private static $_isChunkedDload; #region Public Methods public static function Init(VideoConverter $converter) { self::$_converter = $converter; } public static function ChunkedDownload(array $vars) { extract($vars); self::$_isChunkedDload = true; $converter = self::$_converter; $vHost = $converter->GetCurrentVidHost(); $dloadUrls = end($urls[$vidCount]); $dloadUrls = (!is_array($dloadUrls)) ? array($dloadUrls) : $dloadUrls; foreach ($dloadUrls as $urlKey => $dloadUrl) { self::$_downloaded = self::$_percentVidDownloaded = 0; $dloadUrlInfo = self::CheckDownloadUrl($dloadUrl, $extractor, $vidInfo, $vHost); $dloadUrl = (!empty($dloadUrlInfo['redirectUrl'])) ? $dloadUrlInfo['redirectUrl'] : $dloadUrl; if ($dloadUrlInfo['isValid']) { self::$_fsize = $dloadUrlInfo['filesize']; $chunkEnd = $chunkSize = 1000000; // 1 MB in bytes $numTries = $count = $chunkStart = 0; if (is_file($filename[$urlKey])) unlink($filename[$urlKey]); $file = fopen($filename[$urlKey], 'a'); self::$_curlResource = $ch = curl_init(); while (self::$_fsize >= $chunkStart) { //curl_setopt($ch, CURLOPT_FILE, $file); curl_setopt($ch, CURLOPT_HEADER, 0); curl_setopt($ch, CURLOPT_URL, $dloadUrl); curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); if (Config::_ENABLE_IP_ROTATION && !Config::_DISABLE_IP_FOR_DOWNLOAD && $vHost == "YouTube" && $converter->GetOutgoingIP() != array()) { $currentIP = $converter->GetOutgoingIP(); $isProxy = !empty($currentIP['port']) || !empty($currentIP['proxy_user']) || !empty($currentIP['proxy_pass']); curl_setopt($ch, CURLOPT_REFERER, ''); if ($isProxy) { curl_setopt($ch, CURLOPT_PROXY, $currentIP['ip'] . ":" . $currentIP['port']); if (!empty($currentIP['proxy_user']) && !empty($currentIP['proxy_pass'])) { curl_setopt($ch, CURLOPT_PROXYUSERPWD, $currentIP['proxy_user'] . ":" . $currentIP['proxy_pass']); } if (Config::_ENABLE_TOR_PROXY) { curl_setopt($ch, CURLOPT_PROXYTYPE, CURLPROXY_SOCKS5); } curl_setopt($ch, CURLOPT_TIMEOUT, (int)ceil(3 * (round($chunkSize / 1048576, 2) / (1 / 8)))); } else { curl_setopt($ch, CURLOPT_INTERFACE, $currentIP['ip']); curl_setopt($ch, CURLOPT_IPRESOLVE, CURL_IPRESOLVE_V4); } curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, FALSE); } curl_setopt($ch, CURLOPT_BINARYTRANSFER, 1); curl_setopt($ch, CURLOPT_NOPROGRESS, false); curl_setopt($ch, CURLOPT_PROGRESSFUNCTION, array('self', self::ProgressFuncName())); curl_setopt($ch, CURLOPT_BUFFERSIZE, $chunkSize); curl_setopt($ch, CURLOPT_RANGE, $chunkStart.'-'.$chunkEnd); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt($ch, CURLOPT_USERAGENT, $extractor->GetMainUserAgent()); curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); if (isset($vidInfo['cookies'])) { curl_setopt($ch, CURLOPT_COOKIE, $vidInfo['cookies']); } $output = curl_exec($ch); $curlInfo = curl_getinfo($ch); if ((curl_errno($ch) != 0 || $curlInfo['http_code'] != "206") && $numTries < 10) { $numTries++; continue; } $numTries = 0; fwrite($file, $output); $chunkStart += $chunkSize; $chunkStart += ($count == 0) ? 1 : 0; $chunkEnd += $chunkSize; self::$_chunkCount = ++$count; } curl_close($ch); fclose($file); self::$_prevChunkCount = self::$_chunkCount = 0; } /*if (is_file($filename[$urlKey])) echo "is file: " . $filename[$urlKey] . "<br>";*/ } } public static function Download(array $vars) { extract($vars); self::$_isChunkedDload = false; $converter = self::$_converter; $vHost = $converter->GetCurrentVidHost(); $dloadUrls = end($urls[$vidCount]); $dloadUrls = (!is_array($dloadUrls)) ? array($dloadUrls) : $dloadUrls; $resumeKey = 0; foreach ($dloadUrls as $urlKey => $dloadUrl) { self::$_curlResource = $ch = curl_init(); while ($urlKey == $resumeKey) { self::$_percentVidDownloaded = 0; $file = fopen($filename[$urlKey], 'w'); curl_setopt($ch, CURLOPT_FILE, $file); curl_setopt($ch, CURLOPT_HEADER, 0); curl_setopt($ch, CURLOPT_URL, $dloadUrl); curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); if (Config::_ENABLE_IP_ROTATION && !Config::_DISABLE_IP_FOR_DOWNLOAD && $vHost == "YouTube" && $converter->GetOutgoingIP() != array()) { $currentIP = $converter->GetOutgoingIP(); $isProxy = !empty($currentIP['port']) || !empty($currentIP['proxy_user']) || !empty($currentIP['proxy_pass']); curl_setopt($ch, CURLOPT_REFERER, ''); if ($isProxy) { curl_setopt($ch, CURLOPT_PROXY, $currentIP['ip'] . ":" . $currentIP['port']); if (!empty($currentIP['proxy_user']) && !empty($currentIP['proxy_pass'])) { curl_setopt($ch, CURLOPT_PROXYUSERPWD, $currentIP['proxy_user'] . ":" . $currentIP['proxy_pass']); } if (Config::_ENABLE_TOR_PROXY) { curl_setopt($ch, CURLOPT_PROXYTYPE, CURLPROXY_SOCKS5); } } else { curl_setopt($ch, CURLOPT_INTERFACE, $currentIP['ip']); curl_setopt($ch, CURLOPT_IPRESOLVE, CURL_IPRESOLVE_V4); } curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, FALSE); } curl_setopt($ch, CURLOPT_NOPROGRESS, false); curl_setopt($ch, CURLOPT_PROGRESSFUNCTION, array('self', self::ProgressFuncName())); curl_setopt($ch, CURLOPT_BUFFERSIZE, 4096000); curl_setopt($ch, CURLOPT_USERAGENT, $extractor->GetMainUserAgent()); curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); if (isset($vidInfo['cookies'])) { curl_setopt($ch, CURLOPT_COOKIE, $vidInfo['cookies']); } curl_exec($ch); if (curl_errno($ch) == 0) { $curlInfo = curl_getinfo($ch); if (($vHost == "Dailymotion" || $vHost == "SoundCloud" || $vHost == "YouTube" || $vHost == "Pornhub") && $curlInfo['http_code'] == '302' && isset($curlInfo['redirect_url']) && !empty($curlInfo['redirect_url'])) { $dloadUrl = $curlInfo['redirect_url']; continue; } if (method_exists($extractor, 'GetCypherUsed') && $extractor->GetCypherUsed() && $curlInfo['http_code'] == '403') { $itag = $extractor->ExtractItagFromUrl($dloadUrl); if (!empty($itag)) { $extractor->FixDecryption($extractor->GetSignature($itag)); } } } fclose($file); $resumeKey++; } curl_close($ch); } } public static function DownloadPlaylist(array $vars) { extract($vars); $converter = self::$_converter; $vHost = $converter->GetCurrentVidHost(); $reqHeaders = (!empty($extractor->_reqHeaders)) ? $extractor->_reqHeaders : ""; $cmd = Config::_FFMPEG . (isset($pre_ffmpeg) ? $pre_ffmpeg : '') . ((!empty($reqHeaders)) ? ' -headers ' . escapeshellarg(implode('\r\n', $reqHeaders) . '\r\n') : '') . ' -i \'' . end($urls[$vidCount]) . '\' -bsf:a ' . ((strrchr($filename[0], ".") == ".mp3" || $vHost == "SoundCloud") ? 'mp3decomp' : 'aac_adtstoasc') . ' -c copy -y ' . $filename[0] . ' 2>&1'; //die($cmd); if (Config::_ENABLE_PLAYLIST_DOWNLOAD_PROGRESS && isset($vidInfo['duration'])) { $descriptorspec = array( 0 => array("pipe", "r"), 1 => array("pipe", "w"), 2 => array("pipe", "a") ); $pipes = array(); $process = proc_open($cmd, $descriptorspec, $pipes, null, null); if (is_resource($process)) { $processInfo = false; do { $cmdOutputLine = trim(fgets($pipes[1])); if (preg_match('/(time=)(.+?)(\s)/i', $cmdOutputLine, $times) == 1) { if (preg_match('/(\d\d):(\d\d):(\d\d\.\d\d)/', $times[2], $lastTime) == 1) { $lastTime = ((int)$lastTime[1] * 60 * 60) + ((int)$lastTime[2] * 60) + (float)$lastTime[3]; $progress = round(($lastTime / (float)$vidInfo['duration']) * 100); $progress = ($progress > 100) ? 100 : $progress; self::OutputDownloadProgress($progress, true); } } //echo $cmdOutputLine . "<br>"; if (!empty($cmdOutputLine)) $ffmpegOutput[] = $cmdOutputLine; $processInfo = proc_get_status($process); } while ($processInfo !== false && $processInfo['running']); } fclose($pipes[0]); fclose($pipes[1]); fclose($pipes[2]); proc_close($process); } else { self::OutputDownloadProgress(100, false); exec($cmd, $ffmpegOutput); } return $ffmpegOutput; } public static function DownloadPlaylistNative(array $vars) { extract($vars); $converter = self::$_converter; $vHost = $converter->GetCurrentVidHost(); $reqHeaders = (!empty($extractor->_reqHeaders)) ? $extractor->_reqHeaders : ""; self::OutputDownloadProgress(100, false); $context = stream_context_create(array( 'http' => array( 'method' => "GET", 'header' => $reqHeaders ) )); $m3u8Url = end($urls[$vidCount]); $m3u8file = file_get_contents($m3u8Url, false, $context); if ($m3u8file !== false && !empty($m3u8file)) { $m3u8Lines = preg_split('/\n|\r/', $m3u8file, -1, PREG_SPLIT_NO_EMPTY); $m3u8Lines = preg_grep('/^(#)/', $m3u8Lines, PREG_GREP_INVERT); //die(print_r($m3u8Lines)); if (!empty($m3u8Lines)) { ini_set('memory_limit', '-1'); $videoContent = ''; foreach ($m3u8Lines as $m3u8Line) { //die($m3u8Line); $urlPrefix = preg_replace('/(' . preg_quote(strrchr($m3u8Url, "/"), "/") . ')$/', "", $m3u8Url); $m3u8Line = $urlPrefix . "/" . $m3u8Line; //die($m3u8Line); $tsFileContent = file_get_contents($m3u8Line, false, $context); if ($tsFileContent === false || empty($tsFileContent)) { $videoContent = ''; break; } $videoContent .= $tsFileContent; } if (!empty($videoContent)) { $tmpfname = tempnam(dirname(__DIR__) . "/store", "m3u8"); if ($tmpfname !== false) { $bytes = file_put_contents($tmpfname, $videoContent); if ($bytes !== false && $bytes > 0) { $cmd = Config::_FFMPEG . (isset($pre_ffmpeg) ? $pre_ffmpeg : '') . ' -i ' . escapeshellarg($tmpfname) . ' -c copy -y -f mp4 -bsf:a aac_adtstoasc ' . escapeshellarg($filename[0]) . ' 2>&1'; exec($cmd, $ffmpegOutput); } unlink($tmpfname); } } } } return $ffmpegOutput; } public static function OutputDownloadProgress($percent, $isRealTime) { echo '<script type="text/javascript">updateVideoDownloadProgress("'. $percent .'", ' . (($isRealTime) ? 'true' : 'false') . ');</script>'; $converter = self::$_converter; $converter->FlushBuffer(); } #endregion #region Private "Helper" Methods private static function CheckDownloadUrl($url, Extractor $extractor, array $vidInfo, $vHost) { $retVal = array('isValid' => false, 'filesize' => 0, 'redirectUrl' => ''); $converter = self::$_converter; $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); curl_setopt($ch, CURLOPT_NOBODY, true); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); curl_setopt($ch, CURLOPT_HEADER, true); curl_setopt($ch, CURLOPT_USERAGENT, $extractor->GetMainUserAgent()); if (Config::_ENABLE_IP_ROTATION && !Config::_DISABLE_IP_FOR_DOWNLOAD && $converter->GetCurrentVidHost() == "YouTube" && $converter->GetOutgoingIP() != array()) { $currentIP = $converter->GetOutgoingIP(); $isProxy = !empty($currentIP['port']) || !empty($currentIP['proxy_user']) || !empty($currentIP['proxy_pass']); curl_setopt($ch, CURLOPT_REFERER, ''); if ($isProxy) { curl_setopt($ch, CURLOPT_PROXY, $currentIP['ip'] . ":" . $currentIP['port']); if (!empty($currentIP['proxy_user']) && !empty($currentIP['proxy_pass'])) { curl_setopt($ch, CURLOPT_PROXYUSERPWD, $currentIP['proxy_user'] . ":" . $currentIP['proxy_pass']); } if (Config::_ENABLE_TOR_PROXY) { curl_setopt($ch, CURLOPT_PROXYTYPE, CURLPROXY_SOCKS5); } } else { curl_setopt($ch, CURLOPT_INTERFACE, $currentIP['ip']); curl_setopt($ch, CURLOPT_IPRESOLVE, CURL_IPRESOLVE_V4); } curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, FALSE); } curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); if (isset($vidInfo['cookies'])) { curl_setopt($ch, CURLOPT_COOKIE, $vidInfo['cookies']); } $headers = curl_exec($ch); if (curl_errno($ch) == 0) { $info = curl_getinfo($ch); //die(print_r($info)); $retVal['filesize'] = (int)$info['download_content_length']; if (($vHost == "Dailymotion" || $vHost == "SoundCloud" || $vHost == "YouTube" || $vHost == "Pornhub") && $info['http_code'] == '302' && isset($info['redirect_url']) && !empty($info['redirect_url'])) { $retVal['redirectUrl'] = $info['redirect_url']; } if (method_exists($extractor, 'GetCypherUsed') && $extractor->GetCypherUsed() && $info['http_code'] == '403') { $itag = $extractor->ExtractItagFromUrl($url); if (!empty($itag)) { $extractor->FixDecryption($extractor->GetSignature($itag)); } } else { $retVal['isValid'] = $info['http_code'] != '404' && $info['http_code'] != '403'; } } curl_close($ch); return $retVal; } private static function ProgressFuncName() { return (Config::_PHP_VERSION >= 5.5) ? ((self::$_isChunkedDload) ? 'UpdateVideoChunkDownloadProgress' : 'UpdateVideoDownloadProgress') : 'LegacyUpdateVideoDownloadProgress'; } private static function UpdateVideoDownloadProgress($curlResource, $downloadSize, $downloaded, $uploadSize, $uploaded) { $httpCode = curl_getinfo($curlResource, CURLINFO_HTTP_CODE); if ($httpCode == "200" && $downloadSize > 0) { $percent = round($downloaded / $downloadSize, 2) * 100; if ($percent > self::$_percentVidDownloaded) { self::$_percentVidDownloaded++; self::OutputDownloadProgress($percent, true); } } } private static function UpdateVideoChunkDownloadProgress($curlResource, $downloadSize, $downloaded, $uploadSize, $uploaded) { $httpCode = curl_getinfo($curlResource, CURLINFO_HTTP_CODE); if ($httpCode == "206" && $downloadSize > 0 && self::$_chunkCount != self::$_prevChunkCount) { self::$_prevChunkCount++; self::$_downloaded += $downloadSize; $percent = round(self::$_downloaded / self::$_fsize, 2) * 100; if ($percent > self::$_percentVidDownloaded) { self::$_percentVidDownloaded++; self::OutputDownloadProgress($percent, true); } } } // Deprecated - May be removed in future versions! private static function LegacyUpdateVideoDownloadProgress($downloadSize, $downloaded, $uploadSize, $uploaded) { if (self::$_isChunkedDload) { self::UpdateVideoChunkDownloadProgress(self::$_curlResource, $downloadSize, $downloaded, $uploadSize, $uploaded); } else { self::UpdateVideoDownloadProgress(self::$_curlResource, $downloadSize, $downloaded, $uploadSize, $uploaded); } } #endregion } ?> (03) VideoConverter.php <?php namespace YouTubeMp3Converter\lib; // Conversion Class class VideoConverter extends Config { // Private Fields private $_convertedFileName = ''; private $_convertedFileType = ''; private $_convertedFileCategory = ''; private $_convertedFileQuality = Config::_DEFAULT_AUDIO_QUALITY; private $_convertedFileVolume = Config::_VOLUME; private $_vidSourceUrls = array(); private $_tempVidFileName = array(); private $_uniqueID; private $_currentVidHost = ''; private $_vidInfo = array(); private $_validationError = ''; private $_skipConversion = false; private $_ffmpegCommand = ''; private $_extractor; private $_outgoingIP = array(); private $_doFFmpegCopy = false; private $_pluginInfo = array(); // Constants const _FILENAME_DELIMITER = "~~"; const _MAX_FILENAME_LENGTH = 255; const _URL_WILDCARD_PATTERN = '[^\\\\/\?]+'; #region Public Methods function __construct() { if (isset($_SESSION)) { $this->_uniqueID = (!isset($_SESSION[Config::_SITENAME])) ? time() . "_" . uniqid('', true) : $_SESSION[Config::_SITENAME]; $_SESSION[Config::_SITENAME] = (!isset($_SESSION[Config::_SITENAME])) ? $this->_uniqueID : $_SESSION[Config::_SITENAME]; $_SESSION['execFFmpegToken'] = (!isset($_SESSION['execFFmpegToken'])) ? uniqid($this->_uniqueID, true) : $_SESSION['execFFmpegToken']; $_SESSION['execFFmpegToken2'] = (!isset($_SESSION['execFFmpegToken2'])) ? uniqid($this->_uniqueID, true) : $_SESSION['execFFmpegToken2']; if (Config::_ENABLE_IP_ROTATION && !Config::_ENABLE_TOR_PROXY) Database::Connect(Config::_SERVER, Config::_DB_USER, Config::_DB_PASSWORD, Config::_DATABASE); $this->_pluginInfo = Plugin::Init(); // Load any plugin data } else { die('Error!: Session must be started in the calling file to use this class.'); } } function __destruct() { if (Config::_ENABLE_IP_ROTATION && !Config::_ENABLE_TOR_PROXY && class_exists('Database')) Database::Close(); } function DownloadVideo($vidUrl) { $videoInfo = $this->GetVidInfo(); if (!empty($videoInfo)) { $this->SetConvertedFileName(); $this->SetVidSourceUrls(); if ($this->GetConvertedFileName() != '' && count($this->GetVidSourceUrls()) > 0) { $urls = $this->GetVidSourceUrls(); if ((Config::_CACHING_ENABLED && !Config::_ENABLE_DIRECT_DOWNLOAD) || (Config::_ENABLE_DIRECT_DOWNLOAD && $this->GetConvertedFileCategory() != 'audio' && $this->GetConvertedFileVolume() == Config::_VOLUME)) { $urls = $this->FilterUrls($urls); //die(print_r($urls)); } return $this->SaveVideo($urls); } } return false; } function DoConversion() { $extractor = $this->GetExtractor(); $vidHost = $this->GetCurrentVidHost(); $fileType = $this->GetConvertedFileType(); $fileCategory = $this->GetConvertedFileCategory(); $fileQuality = $this->GetConvertedFileQuality(); $fileVolume = $this->GetConvertedFileVolume(); $newFile = $this->GetConvertedFileName(); $tempFile = $this->GetTempVidFileName(); $tmpNewFile = preg_replace('/^((.+)(' . preg_quote(strrchr($newFile, "."), '/') . '))$/', "$2.tmp$3", $newFile); if (!empty($fileType) && !empty($newFile) && !empty($tempFile)) { $exec_string = ''; $ftypes = $this->GetConvertedFileTypes(); foreach ($ftypes as $ftype) { if ($fileType == $ftype['fileExt']) { $videoBitrate = Config::_DEFAULT_VIDEO_QUALITY; if ($fileCategory == 'video') { exec(Config::_FFMPEG . ' -i ' . $tempFile[0] . ' 2>&1 | grep "Video:\|bitrate:"', $output); if (count($output) > 0) { foreach ($output as $line) { if (preg_match('/(\d+)( kb\/s)/i', $line, $matches) == 1) { $videoBitrate = $matches[1]; break; } } } } $ftypeFFmpeg = (isset($ftype['ffmpegCopy']) && $fileVolume == Config::_VOLUME && $fileQuality == Config::_DEFAULT_AUDIO_QUALITY && (($fileType == "aac" && $vidHost == "YouTube" && $extractor->AudioAvailable()) || $this->_doFFmpegCopy)) ? $ftype['ffmpegCopy'] : $ftype['ffmpeg']; $tempFile2 = ''; if (isset($ftype['ffmpegMerge'], $ftype['ffmpegMergeAndVol']) && count($tempFile) > 1) { $ftypeFFmpeg = ($fileVolume == Config::_VOLUME) ? $ftype['ffmpegMerge'] : $ftype['ffmpegMergeAndVol']; $tempFile2 = $tempFile[1]; } $this->_ffmpegCommand = $exec_string = preg_replace( array('/%ffmpeg%/', '/%tempFile%/', '/%tempFile2%/', '/%volume%/', '/%quality%/', '/%vquality%/', '/%newFile%/', '/%logsDir%/', '/%id%/'), array(Config::_FFMPEG, $tempFile[0], $tempFile2, $fileVolume, $fileQuality, $videoBitrate, $tmpNewFile, Config::_LOGSDIR, $this->_uniqueID), $ftypeFFmpeg ); break; } } //die($exec_string); if (!is_dir(realpath(Config::_LOGSDIR))) mkdir(Config::_LOGSDIR, 0777); if (is_file(realpath(Config::_LOGSDIR . $this->_uniqueID . ".txt"))) unlink(realpath(Config::_LOGSDIR . $this->_uniqueID . ".txt")); // If previous conversion was abandoned, remove corresponding log file with same file name, if it exists, to prevent subsequent conversion failure! $isHttps = (!empty($_SERVER['HTTPS']) && $_SERVER['HTTPS'] != 'off') || $_SERVER['SERVER_PORT'] == 443; if (!$isHttps && isset($_SERVER['HTTP_CF_VISITOR'])) { $cfJson = json_decode($_SERVER['HTTP_CF_VISITOR'], true); if (json_last_error() == JSON_ERROR_NONE) { $isHttps = !empty($cfJson) && current($cfJson) == 'https'; } } $protocol = ($isHttps) ? "https://" : "http://"; $ffmpegExecUrl = preg_replace('/(([^\/]+?)(\.php))$/', "exec_ffmpeg.php", $protocol.$_SERVER['HTTP_HOST'].$_SERVER['PHP_SELF']); $postData = "cmd=".urlencode($exec_string)."&token=".urlencode($_SESSION['execFFmpegToken'])."&fname=".urlencode($tmpNewFile); $strCookie = 'PHPSESSID=' . session_id() . '; path=/'; $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, $ffmpegExecUrl); curl_setopt($ch, CURLOPT_POST, true); curl_setopt($ch, CURLOPT_POSTFIELDS, $postData); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); curl_setopt($ch, CURLOPT_TIMEOUT, 1); curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); curl_setopt($ch, CURLOPT_COOKIE, $strCookie); curl_exec($ch); curl_close($ch); } } function DownloadConvertedFile($file, $directory) { $ftypes = $this->GetConvertedFileTypes(); $filepath = $directory . urldecode($file); $ftype = trim(strrchr($filepath, '.'), '.'); $vidHostAbbr = current(explode('/', urldecode($file))); $mimeTypes = array(); array_walk($ftypes, function($ftype) use(&$mimeTypes) {$mimeTypes[$ftype['fileExt']] = $ftype['mimeType'];}); //print_r($mimeTypes); //die("\n\nftype: " . $ftype . ", vidHostAbbr: " . $vidHostAbbr); if ($this->ValidateDownloadFileName($filepath, $directory, array_keys($mimeTypes))) { $filename = $this->PrepareConvertedFileNameForDownload($file); $filepath = realpath($filepath); if (Config::_DELAY_POPULAR_FILE_DELETION && Config::_CACHING_ENABLED && (!Config::_ENABLE_DIRECT_DOWNLOAD || (Config::_ENABLE_DIRECT_DOWNLOAD && $vidHostAbbr != "sc" && ($ftype == "mp3" || $ftype == "aac")))) { $attr = array(); exec(Config::_ATTR . ' -q -g time.created ' . escapeshellarg($filepath) . ' 2>&1', $attr); if (count($attr) > 1) { $attr = array(); exec(Config::_ATTR . ' -s time.created -V ' . time() . ' ' . escapeshellarg($filepath)); exec(Config::_ATTR . ' -q -g time.created ' . escapeshellarg($filepath) . ' 2>&1', $attr); } if (count($attr) == 1 && preg_match('/^(\d+)$/', $attr[0]) == 1) { if (time() - (int)$attr[0] < Config::_MAX_DELETION_DELAY && time() - filemtime($filepath) < Config::_MAX_POPULAR_FILE_INCREMENT) { touch($filepath); } } } //if (filesize($filepath) > 10000) touch($filepath); $contentType = ($ftype == 'm4a') ? $mimeTypes['mp3'] : $mimeTypes[$ftype]; header('Content-Type: ' . $contentType); header('Content-Length: ' . filesize($filepath)); header('Content-Disposition: attachment; filename="'.$filename.'"'); ob_clean(); flush(); readfile($filepath); die(); } else { $redirect = explode("?", $_SERVER['REQUEST_URI']); header('Location: ' . $redirect[0]); } } function ValidateConversionForm($vidUrl, $ftype, $getVidInfo=false, $moreOptions=array()) { $vidHostName = $convertedFtype = ''; $vidHosts = $this->GetVideoHosts(); $urlRoots = array(); $urlSuffix = ''; foreach ($vidHosts as $host) { foreach ($host['url_root'] as $urlRoot) { //$urlRoot = preg_replace('/^(([^\?]+?)(\?{1})(.+))$/', "$2$3", $urlRoot); $wildcardRegex = (Config::_PHP_VERSION >= 7.3) ? '/\\\#wildcard\\\#/' : '/#wildcard#/'; $rootUrlPattern = preg_replace($wildcardRegex, self::_URL_WILDCARD_PATTERN, preg_quote($urlRoot, '/')); $rootUrlPattern = ($host['allow_https_urls']) ? preg_replace('/^(http)/', "https?", $rootUrlPattern) : $rootUrlPattern; if (preg_match('/^(('.$rootUrlPattern.')(.+))/i', $vidUrl, $matches) == 1 && preg_match('/'.$rootUrlPattern.'/', $matches[3]) != 1) { $vidHostName = $host['name']; $urlRoots = $host['url_root']; $urlSuffix = $matches[3]; break 2; } } } $ftypes = $this->GetConvertedFileTypes(); $convertedFtype = (in_array($ftype, array_keys($ftypes))) ? $ftypes[$ftype]['fileExt'] : ''; $convertedFcategory = (in_array($ftype, array_keys($ftypes))) ? current(explode("/", $ftypes[$ftype]['mimeType'])) : ''; $convertedFquality = (in_array($ftype, array_keys($ftypes)) && isset($ftypes[$ftype]['quality'])) ? $ftypes[$ftype]['quality'] : Config::_DEFAULT_AUDIO_QUALITY; $convertedFvolume = (isset($moreOptions['volume'])) ? $moreOptions['volume'] : Config::_VOLUME; if (!empty($vidHostName) && !empty($convertedFtype) && !empty($convertedFcategory)) { foreach ($urlRoots as $urlroot) { $urlroot = preg_replace('/^(https?)/', "", $urlroot); if (isset(Config::$_urlBlacklist["https" . $urlroot . $urlSuffix]) || isset(Config::$_urlBlacklist["http" . $urlroot . $urlSuffix])) { $this->_validationError = 'Validation_Error_Copyright'; return false; } } if ($vidHostName == 'SoundCloud' && $convertedFcategory == 'video') { $this->_validationError = 'Validation_Error_Audio_To_Video'; return false; } $this->SetCurrentVidHost($vidHostName); $this->SetConvertedFileType($convertedFtype); $this->SetConvertedFileCategory($convertedFcategory); $this->SetConvertedFileQuality($convertedFquality); $this->SetConvertedFileVolume($convertedFvolume); $this->SetExtractor($vidHostName); if ($getVidInfo) { $extractor = $this->GetExtractor(); $this->_vidInfo = $extractor->RetrieveVidInfo($vidUrl); //die(print_r($this->_vidInfo)); if (isset($this->_vidInfo['is_video_audio']) && !$this->_vidInfo['is_video_audio']) { $this->_validationError = 'Validation_Error_General'; return false; } $isOkDuration = true; if (isset($this->_vidInfo['duration'])) { foreach ($ftypes as $ftype) { if ($isOkDuration) { $quality = (isset($ftype['quality'])) ? $ftype['quality'] : Config::_DEFAULT_AUDIO_QUALITY; $isOkDuration = ($ftype['fileExt'] == $convertedFtype && $quality == $convertedFquality) ? (($ftype['maxDuration'] != -1) ? $ftype['maxDuration'] >= $this->_vidInfo['duration'] : true) : true; } } } if (!$isOkDuration) { $this->_validationError = 'Validation_Error_Vid_Length'; return false; } } return true; } $this->_validationError = 'Validation_Error_General'; return false; } function PrepareConvertedFileNameForDownload($file) { $filename = urldecode($file); $filename = current(array_reverse(explode(self::_FILENAME_DELIMITER, $filename))); $ftypesForRegex = $this->GetConvertedFileTypes(); if (Config::_ENABLE_CONCURRENCY_CONTROL) { array_walk($ftypesForRegex, function(&$ftype, $key) {$ftype = $ftype['fileExt'];}); $replacementStr = ((Config::_ENABLE_FILENAME_BRANDING) ? '[' . Config::_SITENAME . ']' : '') . "$4$5"; $filename = preg_replace('/((_uuid-)(\w{13})(\.)('.implode('|', $ftypesForRegex).'))$/', $replacementStr, $filename); } $fileBasename = pathinfo($filename, PATHINFO_FILENAME); $filename = (empty($fileBasename) || (!Config::_ENABLE_UNICODE_SUPPORT && preg_match('/^([^a-zA-Z0-9]+)$/', $fileBasename) == 1)) ? 'unknown' . strrchr($filename, '.') : $filename; return preg_replace('/_/', " ", $filename); } function ExtractVideoId($vidUrl) { $id = ''; $url = trim($vidUrl); $urlQueryStr = parse_url($url, PHP_URL_QUERY); if ($urlQueryStr !== false && !empty($urlQueryStr)) { parse_str($urlQueryStr, $params); if (isset($params['v']) && !empty($params['v'])) { $id = $params['v']; } else { $url = preg_replace('/(\?' . preg_quote($urlQueryStr, '/') . ')$/', "", $url); $id = trim(strrchr(trim($url, '/'), '/'), '/'); } } else { $id = trim(strrchr(trim($url, '/'), '/'), '/'); } return $id; } function RetrieveCachedFile() { $fileName = ''; $videoInfo = $this->GetVidInfo(); $ftype = $this->GetConvertedFileType(); $fquality = $this->GetConvertedFileQuality(); $fvolume = $this->GetConvertedFileVolume(); $extractor = $this->GetExtractor(); $vidHost = $this->GetCurrentVidHost(); $videoInfo['host_abbrev'] = $extractor->ReturnConfig('abbreviation'); if ((!Config::_ENABLE_DIRECT_DOWNLOAD || (Config::_ENABLE_DIRECT_DOWNLOAD && $vidHost != "SoundCloud" && ($ftype == "mp3" || $ftype == "aac"))) && !empty($videoInfo) && !empty($videoInfo['title']) && !empty($videoInfo['id']) && !is_null($videoInfo['host_abbrev'])) { $vTitle = html_entity_decode($videoInfo['title'], ENT_COMPAT | ENT_HTML401, 'UTF-8'); $fname = (!Config::_ENABLE_UNICODE_SUPPORT) ? preg_replace('/[^A-Za-z0-9 _-]/', '', $vTitle) : preg_replace('#/#', '', preg_replace('/\\\\|\/|\?|%|\*|:|\||"|<|>|\]|\[|\(|\)|\.|&|\^|\$|#|@|\!|`|~|=|\+|,|;|\'|\{|\}/', '', $vTitle)); $fname = preg_replace('/_{2,}/', '_', preg_replace('/ /', '_', $fname)); $dirName = Config::_CONVERTED_FILEDIR . $videoInfo['host_abbrev'] . '/' . $videoInfo['id'] . '/'; if (is_dir(realpath($dirName))) { $filesystemIterator = new \FilesystemIterator(realpath($dirName), \FilesystemIterator::KEY_AS_FILENAME); $regexIterator = new \RegexIterator($filesystemIterator, '/^(('.preg_quote($fquality . self::_FILENAME_DELIMITER . $fvolume . self::_FILENAME_DELIMITER . $fname, '/').')((_uuid-)(\w+))?(\.)('.preg_quote($ftype, '/').'))$/', \RegexIterator::MATCH, \RegexIterator::USE_KEY); $files = array_keys(iterator_to_array($regexIterator)); if (!empty($files)) { foreach ($files as $file) { if (is_file(realpath($dirName . $file))) { $fileName = $dirName . $file; break; } } } } } //die($fileName); return $fileName; } function FlushBuffer() { if (ob_get_length() > 0) ob_end_flush(); if (ob_get_length() > 0) ob_flush(); flush(); } function ValidateFile(array $filename, array $params) { extract($params); $error = true; foreach ($filename as $fname) { if (is_file($fname)) { $durationDiff = 0; $getID3 = new \getID3; $fileInfo = @$getID3->analyze($fname); //print_r($fileInfo); if (isset($duration)) { if (!isset($fileInfo['playtime_seconds'])) { // Use FFmpeg to detect duration as a backup!! exec(Config::_FFMPEG . ' -i ' . $fname . ' 2>&1', $ffOutput); //die(print_r($ffOutput)); if (count($ffOutput) > 0) { foreach ($ffOutput as $line) { if (preg_match('/Duration:\s*(\d{2}:\d{2}:\d{2})/', $line, $matches) == 1) { $fileInfo['playtime_seconds'] = strtotime("1970-01-01 " . $matches[1] . " UTC"); break; } } } } $durationDiff = (isset($fileInfo['playtime_seconds'])) ? abs((float)$fileInfo['playtime_seconds'] - (float)$duration) : $durationDiff; } $error = !filesize($fname) || filesize($fname) < 10000 || (isset($isPlaylist) && isset($ffmpegOutput) && $isPlaylist && (empty($ffmpegOutput) || preg_match('/muxing overhead/i', end($ffmpegOutput)) != 1)) || !isset($fileInfo['playtime_seconds']) || $durationDiff > Config::_MAX_ALLOWED_DURATION_DIFFERENCE; if ($error) break; } } return !$error; } #endregion #region Private "Helper" Methods private function ValidateDownloadFileName($filepath, $directory, array $fileExts) { $isValid = false; $fullFilepath = realpath($filepath); if ($fullFilepath !== false && $fullFilepath != $filepath && is_file($fullFilepath)) { $normalizedAppRoot = (Config::_APPROOT != "/") ? preg_replace('/\//', DIRECTORY_SEPARATOR, Config::_APPROOT) : DIRECTORY_SEPARATOR; $pathBase = realpath($_SERVER['DOCUMENT_ROOT']) . $normalizedAppRoot; $safePath = preg_replace('/^(' . preg_quote($pathBase, '/') . ')/', "", $fullFilepath); if ($safePath != $fullFilepath && preg_match('/^(' . preg_quote(preg_replace('/\//', DIRECTORY_SEPARATOR, $directory), '/') . ')/', $safePath) == 1) { $fileExt = pathinfo($fullFilepath, PATHINFO_EXTENSION); $isValid = in_array($fileExt, $fileExts); } } return $isValid; } private function FilterUrls(array $urls) { $filteredUrls = array(); $ftype = $this->GetConvertedFileType(); $vidHosts = array_values($this->GetVideoHosts()); $vidQualities = array_keys($vidHosts[0]['video_qualities']); $ftypes = $this->GetConvertedFileTypes(); $uniqueFtypes = array(); array_walk($ftypes, function($ft, $key) use(&$uniqueFtypes) {if (!isset($uniqueFtypes[$ft['fileExt']]) && isset($ft['qualityTolerance'])) $uniqueFtypes[$ft['fileExt']] = $ft['qualityTolerance'];}); if (Config::_ENABLE_DIRECT_DOWNLOAD && isset($uniqueFtypes[$ftype])) { $availableQualityIndexes = array(); array_walk($urls, function($url) use(&$availableQualityIndexes, $vidQualities) {if (in_array($url[1], $vidQualities)) $availableQualityIndexes[] = array_search($url[1], $vidQualities);}); $ftypeQualityIndex = array_search($uniqueFtypes[$ftype], $vidQualities); $reduceQualityToleranceFurther = false; do { $filteredAvailableQuals = array_filter($availableQualityIndexes, function($index) use($ftypeQualityIndex) {return $index <= $ftypeQualityIndex;}); if (empty($filteredAvailableQuals)) { $uniqueFtypes[$ftype] = $vidQualities[++$ftypeQualityIndex]; $reduceQualityToleranceFurther = $ftypeQualityIndex < count($vidQualities) - 1; } else { $reduceQualityToleranceFurther = false; } } while ($reduceQualityToleranceFurther); } foreach ($urls as $url) { $qualityToleranceCondition = (Config::_ENABLE_DIRECT_DOWNLOAD) ? array_search($url[1], $vidQualities) <= array_search($uniqueFtypes[$ftype], $vidQualities) : true; if ($ftype == $url[0] && in_array($url[1], $vidQualities) && $qualityToleranceCondition) { $filteredUrls[] = $url; } } return $filteredUrls; } private function SaveVideo(array $urls) { //die(print_r($urls)); //die(print_r($this->GetVidSourceUrls())); $vidInfo = $this->GetVidInfo(); $extractor = $this->GetExtractor(); $this->_skipConversion = $skipConversion = Config::_ENABLE_DIRECT_DOWNLOAD && (($this->GetConvertedFileCategory() != 'audio' && ($this->GetCurrentVidHost() != 'YouTube' || ($this->GetConvertedFileType() == '3gp' && $extractor->ThreegpAvailable()))) || ($this->GetCurrentVidHost() == 'SoundCloud' && !$vidInfo['downloadable'] && $this->GetConvertedFileType() == 'mp3' && $this->GetConvertedFileQuality() == '128') || ($this->GetCurrentVidHost() == 'YouTube' && $extractor->AudioAvailable() && $this->GetConvertedFileType() == 'm4a')) && $this->GetConvertedFileVolume() == Config::_VOLUME && !empty($urls); $this->_doFFmpegCopy = $doFFmpegCopy = ((Config::_CACHING_ENABLED && !Config::_ENABLE_DIRECT_DOWNLOAD) || (Config::_ENABLE_DIRECT_DOWNLOAD && $this->GetCurrentVidHost() == 'YouTube' && $this->GetConvertedFileCategory() != 'audio' && !$skipConversion)) && !empty($urls); if (!$skipConversion && !$doFFmpegCopy) $urls = $this->GetVidSourceUrls(); $success = false; $vidCount = -1; //die(print_r($urls)); $urls = array_values($urls); while (!$success && ++$vidCount < count($urls)) { if (isset($urls[$vidCount-1]) && $urls[$vidCount-1][1] == 'au' && $skipConversion && $this->GetCurrentVidHost() == 'YouTube' && $this->GetConvertedFileType() == 'm4a') { $this->_skipConversion = $skipConversion = false; } if (!$skipConversion) { $this->SetTempVidFileName(); if (is_array(end($urls[$vidCount]))) { $this->SetTempVidFileName(); } } $filename = (!$skipConversion) ? $this->GetTempVidFileName() : $this->GetConvertedFileName(); $filename = (!is_array($filename)) ? array($filename) : $filename; $tries = 0; $isPlaylist = preg_match('/^((\.m3u8)(.*))$/', (string)strrchr((string)parse_url(end($urls[$vidCount]), PHP_URL_PATH), ".")) == 1; $ffmpegOutput = array(); $pre_ffmpeg = isset($vidInfo['ff_pre']) ? $vidInfo['ff_pre'] : ''; do { $dloadVars = compact('extractor', 'vidInfo', 'urls', 'vidCount', 'filename', 'tries', 'ffmpegOutput', 'pre_ffmpeg'); $remote = (isset($this->_pluginInfo['AntiCaptcha']['Remote'][$this->GetCurrentVidHost()])) ? $this->_pluginInfo['AntiCaptcha']['Remote'][$this->GetCurrentVidHost()] : __NAMESPACE__ . '\\Remote'; $remote::Init($this); if ($isPlaylist) { $nativePlaylistDload = $extractor->ReturnConfig("enable_native_playlist_download"); if(isset($vidInfo['ff_for']) && $vidInfo['ff_for']) $ffmpegOutput = $remote::DownloadPlaylist($dloadVars); else $ffmpegOutput = (!is_null($nativePlaylistDload) && $nativePlaylistDload) ? $remote::DownloadPlaylistNative($dloadVars) : $remote::DownloadPlaylist($dloadVars); //die(print_r($ffmpegOutput)); } else { $isChunkedDload = $extractor->ReturnConfig('enable_chunked_download'); if (!is_null($isChunkedDload) && $isChunkedDload) { $remote::ChunkedDownload($dloadVars); } else { $remote::Download($dloadVars); } } $vidDurArr = (isset($vidInfo['duration'])) ? array('duration' => $vidInfo['duration']) : array(); $success = $this->ValidateFile($filename, $vidDurArr + compact('isPlaylist', 'ffmpegOutput')); if (!$success) { foreach ($filename as $fname) { if (is_file($fname)) unlink($fname); } $ffmpegOutput = array(); } $tries++; } while (!$success && Config::_ENABLE_IP_ROTATION && $tries < Config::_MAX_CURL_TRIES && $this->GetCurrentVidHost() == "YouTube"); } return $success; } #endregion #region Properties public function GetConvertedFileName() { return $this->_convertedFileName; } private function SetConvertedFileName() { $videoInfo = $this->GetVidInfo(); //die($videoInfo['title']); $ftype = $this->GetConvertedFileType(); $fquality = $this->GetConvertedFileQuality(); $fvolume = $this->GetConvertedFileVolume(); $extractor = $this->GetExtractor(); $videoInfo['host_abbrev'] = $extractor->ReturnConfig('abbreviation'); if (!empty($videoInfo) && !empty($videoInfo['title']) && !empty($videoInfo['id']) && !is_null($videoInfo['host_abbrev']) && !empty($ftype)) { $vTitle = html_entity_decode($videoInfo['title'], ENT_COMPAT | ENT_HTML401, 'UTF-8'); $fnameTitle = (!Config::_ENABLE_UNICODE_SUPPORT) ? preg_replace('/[^A-Za-z0-9 _-]/', '', $vTitle) : preg_replace('#/#', '', preg_replace('/\\\\|\/|\?|%|\*|:|\||"|<|>|\]|\[|\(|\)|\.|&|\^|\$|#|@|\!|`|~|=|\+|,|;|\'|\{|\}/', '', $vTitle)); $fnameTitle = preg_replace('/_{2,}/', '_', preg_replace('/ /', '_', $fnameTitle)); $fname = ''; $excessFilenameLength = -1; do { $fnameTitle = ($excessFilenameLength >= 0) ? substr($fnameTitle, 0, strlen($fnameTitle) - $excessFilenameLength - 1) : $fnameTitle; $fname = $fquality . self::_FILENAME_DELIMITER . $fvolume . self::_FILENAME_DELIMITER . $fnameTitle; $fname .= (Config::_ENABLE_CONCURRENCY_CONTROL) ? uniqid('_uuid-') : ''; $fname .= '.' . $ftype; $excessFilenameLength = strlen($fname) - self::_MAX_FILENAME_LENGTH; } while ($excessFilenameLength >= 0); // If file name length is greater than or equal to _MAX_FILENAME_LENGTH bytes, truncate X characters from end of title in file name until the full file name is less than _MAX_FILENAME_LENGTH bytes. $dirName = Config::_CONVERTED_FILEDIR . $videoInfo['host_abbrev'] . '/' . $videoInfo['id'] . '/'; if (!is_dir(realpath($dirName))) mkdir($dirName, 0777, true); $this->_convertedFileName = $dirName . $fname; } //die($this->_convertedFileName); } public function GetVidSourceUrls() { return $this->_vidSourceUrls; } private function SetVidSourceUrls() { $extractor = $this->GetExtractor(); $this->_vidSourceUrls = $extractor->ExtractVidSourceUrls(); } public function GetTempVidFileName() { return $this->_tempVidFileName; } private function SetTempVidFileName() { $extractor = $this->GetExtractor(); $srcVideoType = $extractor->ReturnConfig('src_video_type'); if (!is_null($srcVideoType)) { if (!is_dir(realpath(Config::_TEMPVIDDIR))) mkdir(Config::_TEMPVIDDIR, 0777); $tmpFnameCount = (!empty($this->_tempVidFileName)) ? count($this->_tempVidFileName) + 1 : 1; $tmpFileName = Config::_TEMPVIDDIR . $tmpFnameCount . '_' . $this->_uniqueID . '.' . $srcVideoType; $this->_tempVidFileName = (!empty($this->_tempVidFileName)) ? array_merge($this->_tempVidFileName, array($tmpFileName)) : array($tmpFileName); } //die($this->_tempVidFileName); } public function GetUniqueID() { return $this->_uniqueID; } public function GetConvertedFileTypes() { return $this->_convertedFileTypes; } public function GetVideoHosts() { return $this->_videoHosts; } public function GetCurrentVidHost() { return $this->_currentVidHost; } public function SetCurrentVidHost($hostName) { $this->_currentVidHost = $hostName; } public function GetVidInfo() { return $this->_vidInfo; } public function SetVidInfo($vidInfo) { $this->_vidInfo = $vidInfo; } public function GetConvertedFileType() { return $this->_convertedFileType; } private function SetConvertedFileType($ftype) { $this->_convertedFileType = $ftype; } public function GetConvertedFileCategory() { return $this->_convertedFileCategory; } private function SetConvertedFileCategory($fcat) { $this->_convertedFileCategory = $fcat; } public function GetConvertedFileQuality() { return $this->_convertedFileQuality; } private function SetConvertedFileQuality($quality) { $this->_convertedFileQuality = $quality; } public function GetConvertedFileVolume() { return $this->_convertedFileVolume; } private function SetConvertedFileVolume($volume) { $this->_convertedFileVolume = $volume; } public function GetExtractor() { return $this->_extractor; } public function SetExtractor($vidHostName) { //die(print_r($this->_pluginInfo)); $className = (isset($this->_pluginInfo['AntiCaptcha']['Extractors'][$vidHostName])) ? $this->_pluginInfo['AntiCaptcha']['Extractors'][$vidHostName] : __NAMESPACE__ . "\\extractors\\"; $className .= $vidHostName; try {$this->_extractor = new $className($this);} catch(\Exception $ex) {} } public function GetSkipConversion() { return $this->_skipConversion; } public function GetFFmpegCommand() { return $this->_ffmpegCommand; } public function GetValidationError() { return $this->_validationError; } public function GetOutgoingIP() { return $this->_outgoingIP; } public function SetOutgoingIP() { $noTor = !Config::_ENABLE_TOR_PROXY; $skipIP = false; $outgoingIP = (!$noTor) ? array('ip' => '127.0.0.1', 'port' => Config::_TOR_PROXY_PORT) : array(); $tries = 0; $resetBan = array(); do { if ($noTor) { $resetBan = array(); if (Config::_IP_ROTATION_METHOD == "round-robin") { $ips = Database::Find(Config::_DB_IPS_TABLE, array('order' => array('usage_count'))); $outgoingIP = (!empty($ips)) ? $ips[0] : array(); } else { $ips = Database::Find(Config::_DB_IPS_TABLE, array('order' => array('id'))); $allBanned = true; if (!empty($ips)) { foreach ($ips as $ip) { if ($ip['banned'] == 0) { $outgoingIP = $ip; $allBanned = false; break; } } if ($allBanned) { Database::UpdateAll(Config::_DB_IPS_TABLE, array('banned' => 0)); $ips = Database::Find(Config::_DB_IPS_TABLE, array('order' => array('id'))); $outgoingIP = (!empty($ips)) ? $ips[0] : array(); } } } } if (!empty($outgoingIP)) { if ($this->GetCurrentVidHost() == "YouTube") { $skipIP = ($noTor) ? $outgoingIP['banned'] != 0 && time() - $outgoingIP['banned'] < Config::_IP_BAN_PAUSE : false; if (!$skipIP) { $extractor = $this->GetExtractor(); $ipReqResult = $extractor->CheckIp($outgoingIP); $resetBan = ($noTor) ? (($ipReqResult['isBanned']) ? array('banned' => time()) : array('banned' => 0)) : $resetBan; $skipIP = $ipReqResult['isBanned'] || $ipReqResult['isCurlErr']; if (!$noTor && $skipIP) { $fp = fsockopen($outgoingIP['ip'], Config::_TOR_CONTROL_PORT, $error_number, $err_string, 10); if ($fp !== false) { fwrite($fp, "AUTHENTICATE \"" . Config::_TOR_PROXY_PASSWORD . "\"\n"); $received = fread($fp, 512); fwrite($fp, "signal NEWNYM\n"); $received = fread($fp, 512); fclose($fp); } } } if ($noTor) { Database::Save(Config::_DB_IPS_TABLE, array('id' => $outgoingIP['id'], 'usage_count' => ++$outgoingIP['usage_count']) + $resetBan); } } } $tries++; } while ((empty($outgoingIP) || $skipIP) && $tries < Config::_MAX_CURL_TRIES); $this->_outgoingIP = (empty($outgoingIP)) ? array('ip' => $_SERVER['SERVER_ADDR']) : $outgoingIP; } #endregion } ?> (04) Extractor.php <?php namespace YouTubeMp3Converter\lib\extractors; use YouTubeMp3Converter\lib\Config; use YouTubeMp3Converter\lib\VideoConverter; // Extraction Base Class abstract class Extractor { // Common Fields protected $_converter; protected $_isCurlError = false; protected $_headers = array(); protected $_mainUserAgent = 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.19 (KHTML, like Gecko) Ubuntu/12.04 Chromium/18.0.1025.168 Chrome/18.0.1025.168 Safari/535.19'; protected $_videoWebpageUrl = ''; protected $_videoWebpage = ''; public $_reqHeaders = array(); // Common Public Methods function __construct(VideoConverter $converter) { $this->_converter = $converter; } function ReturnConfig($setting) { $config = NULL; $converter = $this->GetConverter(); $vidHosts = $converter->GetVideoHosts(); foreach ($vidHosts as $host) { if ($host['name'] == $converter->GetCurrentVidHost() && isset($host[$setting])) { $config = $host[$setting]; break; } } return $config; } function CheckIp($ip) { $noWebpageUrl = empty($this->_videoWebpageUrl); $url = ($noWebpageUrl) ? current($this->ReturnConfig('url_root')) . $this->ReturnConfig('url_example_suffix') : $this->_videoWebpageUrl; $ipReqResult = array("isCurlErr" => false, "isBanned" => false); $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, $url); if ($noWebpageUrl) { curl_setopt($ch, CURLOPT_NOBODY, true); curl_setopt($ch, CURLOPT_HEADER, true); } else { curl_setopt($ch, CURLOPT_HEADER, 0); } curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); curl_setopt($ch, CURLOPT_USERAGENT, $this->GetMainUserAgent()); // Set IP options $isProxy = !empty($ip['port']) || !empty($ip['proxy_user']) || !empty($ip['proxy_pass']); curl_setopt($ch, CURLOPT_REFERER, ''); if ($isProxy) { curl_setopt($ch, CURLOPT_PROXY, $ip['ip'] . ":" . $ip['port']); if (!empty($ip['proxy_user']) && !empty($ip['proxy_pass'])) { curl_setopt($ch, CURLOPT_PROXYUSERPWD, $ip['proxy_user'] . ":" . $ip['proxy_pass']); } if (Config::_ENABLE_TOR_PROXY) { curl_setopt($ch, CURLOPT_PROXYTYPE, CURLPROXY_SOCKS5); } curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, Config::_IP_CONNECT_TIMEOUT); curl_setopt($ch, CURLOPT_TIMEOUT, Config::_IP_REQUEST_TIMEOUT); } else { curl_setopt($ch, CURLOPT_INTERFACE, $ip['ip']); curl_setopt($ch, CURLOPT_IPRESOLVE, CURL_IPRESOLVE_V4); } curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, FALSE); curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); $output = curl_exec($ch); $ipReqResult['isCurlErr'] = curl_errno($ch) != 0; if (curl_errno($ch) == 0) { $this->_videoWebpage = (!$noWebpageUrl) ? $output : $this->_videoWebpage; if (!$noWebpageUrl && !empty($output)) { $ipReqResult['isBanned'] = $this->ReturnConfig('name') == "YouTube" && preg_match(YouTube::_CAPTCHA_PATTERN, $output) == 1; } $info = curl_getinfo($ch); //die(print_r($info)); $ipReqResult['isBanned'] = (!$ipReqResult['isBanned']) ? $info['http_code'] == '429' : $ipReqResult['isBanned']; } curl_close($ch); return $ipReqResult; } // Common Protected Methods protected function FileGetContents($url, $postData='', $reqHeaders=array()) { $converter = $this->GetConverter(); if ($converter->GetCurrentVidHost() == "YouTube") { $urlRoot = $this->ReturnConfig('url_root'); $urlRoot = preg_replace('/^(https?)/', "https", $urlRoot[0]); $this->_videoWebpageUrl = (preg_match('/^(' . preg_quote($urlRoot, "/") . ')/', $url) == 1) ? $url : ''; } $file_contents = ''; $tries = 0; do { $this->_headers = array(); $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_HEADER, 0); curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt($ch, CURLOPT_USERAGENT, $this->GetMainUserAgent()); if (Config::_ENABLE_IP_ROTATION && in_array($converter->GetCurrentVidHost(), array("YouTube", "GoogleDrive", "TikTok"))) { if ($converter->GetOutgoingIP() == array() || $tries > 0) $converter->SetOutgoingIP(); if (!empty($this->_videoWebpageUrl) && !empty($this->_videoWebpage)) { return $this->_videoWebpage; } $currentIP = $converter->GetOutgoingIP(); $isProxy = !empty($currentIP['port']) || !empty($currentIP['proxy_user']) || !empty($currentIP['proxy_pass']); curl_setopt($ch, CURLOPT_REFERER, ''); if ($isProxy) { curl_setopt($ch, CURLOPT_PROXY, $currentIP['ip'] . ":" . $currentIP['port']); if (!empty($currentIP['proxy_user']) && !empty($currentIP['proxy_pass'])) { curl_setopt($ch, CURLOPT_PROXYUSERPWD, $currentIP['proxy_user'] . ":" . $currentIP['proxy_pass']); } if (Config::_ENABLE_TOR_PROXY) { curl_setopt($ch, CURLOPT_PROXYTYPE, CURLPROXY_SOCKS5); } curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, Config::_IP_CONNECT_TIMEOUT); curl_setopt($ch, CURLOPT_TIMEOUT, Config::_IP_REQUEST_TIMEOUT); } else { curl_setopt($ch, CURLOPT_INTERFACE, $currentIP['ip']); curl_setopt($ch, CURLOPT_IPRESOLVE, CURL_IPRESOLVE_V4); } curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, FALSE); } if (!empty($postData)) { curl_setopt($ch, CURLOPT_POST, true); curl_setopt($ch, CURLOPT_POSTFIELDS, $postData); } if (!empty($reqHeaders)) { curl_setopt($ch, CURLOPT_HTTPHEADER, $reqHeaders); } curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE); curl_setopt($ch, CURLOPT_HEADERFUNCTION, array($this, 'AppendHttpHeader')); $file_contents = curl_exec($ch); $this->_isCurlError = curl_errno($ch) != 0; $curlInfo = curl_getinfo($ch); //print_r($curlInfo); if (curl_errno($ch) == 0) { if ($converter->GetCurrentVidHost() == "YouTube" && ($curlInfo['http_code'] == '302' || $curlInfo['http_code'] == '301')) { if (isset($curlInfo['redirect_url']) && !empty($curlInfo['redirect_url'])) { $file_contents = $this->FileGetContents($curlInfo['redirect_url']); } } } curl_close($ch); $tries++; } while (Config::_ENABLE_IP_ROTATION && in_array($converter->GetCurrentVidHost(), array("YouTube", "GoogleDrive", "TikTok")) && $tries < Config::_MAX_CURL_TRIES && ($this->_isCurlError || $curlInfo['http_code'] == '403' || $curlInfo['http_code'] == '429' || empty($file_contents) || preg_match(YouTube::_CAPTCHA_PATTERN, $file_contents) == 1)); return $file_contents; } protected function AppendHttpHeader($ch, $headr) { $this->_headers[] = $headr; return strlen($headr); } protected function ExtractCookies() { $cookies = ''; $cookieNames = array(); $headers = array_reverse($this->_headers); foreach ($headers as $headr) { $cookies .= (preg_match('/^(Set-Cookie:\s*(\w+)=([^;]+))/i', $headr, $matches) == 1 && !in_array($matches[2], $cookieNames)) ? $matches[2] . "=" . $matches[3] . ";" : ''; $cookieNames[] = $matches[2]; } return trim($cookies, ";"); } // Force child classes to define these methods abstract public function RetrieveVidInfo($vidUrl); abstract public function ExtractVidSourceUrls(); // Common Properties protected function GetConverter() { return $this->_converter; } public function GetMainUserAgent() { return $this->_mainUserAgent; } public function GetVideoWebpage() { return $this->_videoWebpage; } protected function GetStoreDir() { return dirname(dirname(__DIR__)) . DIRECTORY_SEPARATOR . 'store' . DIRECTORY_SEPARATOR; } } ?> Hope i can get some help to solve this issue. Thank you kindly.

I want to extract current data from a particular website(http://www.ncdex.com/MarketData/LiveFuturesQuotes.aspx) and then show it on my website. I want to extract only if the Data is "Chana" or else move to next line or stop. Because the website keeps on updating and the field(Chana) always changes its rows. Please help me. <html> <body> <?php $html = file_get_contents('http://www.ncdex.com/MarketData/LiveFuturesQuotes.aspx'); //get the html returned from the following url libxml_use_internal_errors( true); $doc = new DOMDocument; $doc->loadHTML( $html); $xpath = new DOMXpath( $doc); $node4a = $xpath->query( '//*[@id="ctl00_ContentPlaceHolder3_dgLiveFuturesQuotes"]/tbody/tr[11]/td[1]/a')->item( 0); $node4b = $xpath->query( '//*[@id="ctl00_ContentPlaceHolder3_dgLiveFuturesQuotes"]/tbody/tr[11]/td[7]')->item( 0); $node4c = $xpath->query( '//*[@id="ctl00_ContentPlaceHolder3_dgLiveFuturesQuotes"]/tbody/tr[11]/td[8]')->item( 0); $src4 = $xpath->query( '//*[@id="ctl00_ContentPlaceHolder3_dgLiveFuturesQuotes"]/tbody/tr[11]/td[7]/img/@src')->item( 0); $node4d = $xpath->query( '//*[@id="ctl00_ContentPlaceHolder3_dgLiveFuturesQuotes"]/tbody/tr[11]/td[2]')->item( 0); echo $node4a->textContent; // This will print **Chana** echo $node4b->textContent; echo $node4c->textContent; echo $img1; echo $src4->nodeValue; echo $img2; echo "|"; echo $node4d->textContent; echo "<br>"; ?> </html>