Slow cURL

[devil6600]

Hello,

 

this is my first post here.

 

why is it so that when i use curl just for a single time fetching it works fine but when used it in loop its quite slow.

For example, I am having a from in which user feeds the information about how many times the data will be fetched from a particular website, if in this form a single time is chooses, the data is fetched and shown immediately, whereas if multiple times (about 50 iterations) is chosen the cURL fetching is very slow.

I am using ajax and php. User fills the ajax form telling how many times the cURL php file needs to be run.

Can anyone help me with this?

 

Thanks

[mrMarcus]

you answered your own question by stating things slow down with more connections than just one.  that's pretty much the case with anything.

 

there's not much you can do.  one of something will usually always execute much faster than 50 of something, just the way it is.

[devil6600]

you answered your own question by stating things slow down with more connections than just one.  that's pretty much the case with anything.

 

there's not much you can do.  one of something will usually always execute much faster than 50 of something, just the way it is.

 

Thanks mrmarcus for your response, actually what i mean in suppose for a single iteration cURL takes 1 second then for 50 iterations it should take 50 seconds or so but it takes around 5-7 minutes to completely execute the script.

So i want the script to run in much less time, maybe for 50 iterations it should take not more than 1 minute.

I hope you get my question.

[mrMarcus]

is it possible for you to just fetch the data one time and loop through the data on your own server?

 

so, instead of doing multiple cURL connections, just do one?  or do you really need to do multiple connections.

 

can you post your code with explanation of your code.

[devil6600]

here's the code.

this is not the actual code but i am doing the same thing. instead of the function call in php i am using ajax to loop the function call.

thanks

<?php
function fetch($data,$url) {
	$params = "enrollNo=$data&Submit=Submit";
	$user_agent = "IE 5.0";
	$ch = curl_init() or die(curl_error());
	curl_setopt($ch, CURLOPT_POST, 1);
	curl_setopt($ch, CURLOPT_POSTFIELDS, $params);
	curl_setopt($ch, CURLOPT_URL, $url);
	curl_setopt($ch, CURLOPT_USERAGENT, $user_agent);
	curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
	$result = curl_exec($ch);
	curl_close($ch);
	print_r($result);
}
for($i=0; $i<=100; $i++) {
	fetch($i, "some_IP_here");
}
?>

[mrMarcus]

i take it you're fetching information based on inputted IP address?  have you tried fetching ALL data, and checking the data on your end against the specific IP address?  see if that makes a difference.

 

i'm not a cURL expert at all, but don't forget, you're cURL applications are only as fast as the two servers sending/receiving the data.