Jump to content

Error w/ cURL (lots of URLs)


Jbudone

Recommended Posts

Hey guys, I'm having a strange problem with cURL that I'm drawing an absolute blank to. It works completely fine locally, but when I upload the files to my webhost and test it over there, it doesn't work at all - it loads for about 2 minutes, and then displays:

 

Server error.

 

The website encountered an error while retrieving http://www.craigwatcher.me/playground/units.php. It may be down for maintenance or configured incorrectly.

 

 

 

 

I've checked the error logs but nothing new is being added into there. Basically I'm just iterating through a list of URLs and cURLing each of them. Let me show you the code that I'm working with:

 



ini_set('max_execution_time',0);

require_once('../system/utilities.php'); // This will give us our getLocations() function, which has been tested to work 100% both locally and online


$categories = array('jjj', 'ggg', 'bbb', 'sss', 'hhh');
$locations = getLocations();
if (!$locations)
exit;
$ch = curl_init(); 
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE); 
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($ch, CURLOPT_TIMEOUT, 10);


foreach ($locations as $country => $cities)
{
$countryCode = getCountryCode($country);

foreach ($cities as $city) {
	foreach ($categories as $category)
	{
		$url=$city.'.en.craigslist.'.$countryCode.'/'.$category.'/index.rss';
echo $url.'<br/>';
		curl_setopt($ch, CURLOPT_URL, $url);
		$rss = curl_exec($ch);
		if (!$rss)
			echo ' FAILED to load: '.$url.'<br/>';
	}
}
}

curl_close($ch);

 

 

 

I've already tested everything else (eg. the getLocations() and getCountryCode(), and they prove to work 100% fine both locally and online. What could it POSSIBLY be!? I'm pulling my hair out over here, my mind is boggling and I'm completely lost at what could possibly be going wrong.

Link to comment
https://forums.phpfreaks.com/topic/224495-error-w-curl-lots-of-urls/
Share on other sites

Gonna take a guess here.

 

If works fine locally but not at the new server then must be your host somehow.

Does host have limits, is server actually live, it seems more like a 500 internal error.

 

Whatever http://www.craigwatcher.me/playground/units.php is times out or I should say just sits there forever in my browser doing nothing.

Archived

This topic is now archived and is closed to further replies.

×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.