Detect Variables

[fanfavorite]

I am trying to get information from a particular website and I can output the entire website, however one section seems to display differently when I use cURL compared to when I go directly to the URL.  The code is: 

 

<?
$URL1 = "http://www.domain.com/"; //ASP Website
$ch = curl_init(); 
curl_setopt($ch, CURLOPT_COOKIEJAR, "cookie.txt");
curl_setopt($ch, CURLOPT_COOKIEFILE, "cookie.txt");
curl_setopt($ch, CURLOPT_RETURNTRANSFER,1);  
curl_setopt($ch, CURLOPT_TIMEOUT,3);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION,1); 
curl_setopt($ch, CURLOPT_URL,$URL1); 

if ($buffer = curl_exec($ch)) {
echo $buffer;
}     
curl_close ($ch); 
?>

 

Is there a way to figure out what might be causing the different data to display? 

[ratcateme]

how is it different?

 

Scott.

[fanfavorite]

Try going to http://www.machosports.com/VwSportSelection.asp and then seeing what http://www.machosports.com/VwSportSelection.asp shows.  Just the middle content is what is changed.  Radio buttons for the middle content through curl, when they are href for other.  It also doesn't have one of the key variables tmpPeriod. 

[ratcateme]

i think you might have posted the wrong link i can't see any radio buttons

 

Scott.

[fanfavorite]

Heres the real link:  http://www.machosports.com/VwSportSelection.asp

 

Heres the link that we have:  http://www.anothertakestudios.com/test.php

[samshel]

can you please explain more what the problem is?

[Third_Degree]

Well clearly somewhere in the source a tag is looking for css/javascript/something from another page on their original server which is not found on yours. You'll need to parse the html somehow and then make another request to the site for the css or whatever it is.

[ratcateme]

you say you got this using cURL but when i look at the source of the two pages they are total different?

 

Scott.

[fanfavorite]

The only part of the code that is different is the middle form "SportSelectionForm".  This is what I don't understand, so Third_Degree is right, but I don't know how to fix this.  I thought that cURL reads it as if you were to just go to the website.  The only other thing that is in my code that is not listed above is some str_replaces to add the full url.  They are irrelevent to post because if I leave them out, it still shows the middle section how it is, just doesn't have any of the other layout proper. 

 

What is displayed would be fine if I could get the tmpPeriod value, which does not display on the cURL page. 

 

I don't really know where to go from here to get that.  file_get_contents is normally what I would use, but for some reason some asp pages give you a 500 Internal Server Error with that.   

[fanfavorite]

http://www.anothertakestudios.com/test1.php is without the str_replaces.  It is exactly as my first post in this topic. 

[Third_Degree]

This as I said is a big job.

 

Easiest way to do it is look in the original page source to see where it needs the css or whatever, and str_replace that another page on your server that forms a curl request to that page.

[fanfavorite]

I already did that.  That is http://www.anothertakestudios.com/test.php.  Unless I am not understanding you correctly: 

 

$URL1 = "http://www.machosports.com/VwSportSelection.asp";

$ch = curl_init();

curl_setopt($ch, CURLOPT_COOKIEJAR, "cookie.txt");

curl_setopt($ch, CURLOPT_COOKIEFILE, "cookie.txt");

curl_setopt($ch, CURLOPT_RETURNTRANSFER,1); 

curl_setopt($ch, CURLOPT_TIMEOUT,3);

curl_setopt($ch, CURLOPT_FOLLOWLOCATION,1);

curl_setopt($ch, CURLOPT_URL,$URL1);

 

if ($buffer = curl_exec($ch)) {

preg_match_all("/.sportType.value = '(.*)'/Uis",$buffer, $matches);

preg_match_all("/.sportSubType.value = '(.*)'/Uis",$buffer, $matches2);

preg_match_all("/.tmpPeriod.value = '(.*)'/Uis",$buffer, $matches3);

$counter = 0;

foreach ($matches[1] as $match) {

//echo $counter.": ".$match;

//echo ",".$matches2[1][$counter];

//echo ",".$matches3[1][$counter];

//echo "<br />";

$counter++;

}

$buffer = str_replace('href="','href="http://www.machosports.com',$buffer);

$buffer = str_replace('name="movie" value="','name="movie" value="http://www.machosports.com/',$buffer);

$buffer = str_replace('embed src="','embed src="http://www.machosports.com/',$buffer);

$buffer = str_replace('script src="','script src="http://www.machosports.com',$buffer);

$buffer = str_replace('myFlashObject.movie="','myFlashObject.movie="http://www.machosports.com/',$buffer);

$buffer = str_replace('IMG SRC="','IMG SRC="http://www.machosports.com/',$buffer);

$buffer = str_replace('img src="','img src="http://www.machosports.com/',$buffer);

$buffer = str_replace('type="text/javascript" src="','type="text/javascript" src="http://www.machosports.com/',$buffer);

$buffer = str_replace('cellspacing="0" background="','cellspacing="0" background="http://www.machosports.com/',$buffer);

$buffer = str_replace('type="image" src="','type="image" src="http://www.machosports.com/',$buffer);

echo $buffer;

}   

curl_close ($ch);

[Third_Degree]

<link href="/inc/css/site.css" media="screen" type="text/css" rel="stylesheet">

 

That line's your problem I think.

[fanfavorite]

I don't see how the CSS will change source code.  It's not the overall look that I care about, its the actual data.  The only thing CSS can do is hide layers.  Now I reviewed that site.css and there is nothing in there with any significance to my problem.  It is linked fine in the curl request too.   

[fanfavorite]

My thought is there must be a variable in js or asp that is selecting how to output the text or there is a problem with the PHP cURL for displaying this page.  Not sure how to detect this, if it even is possible. 

[Third_Degree]

Well if the server's deciding what to show you, there's not much you can do. Also, I copied the source of both of the pages (original and test.php) and ran a comparison script and they were the same...