Jump to content

[SOLVED] Why does my crawler script suddenly end with no error?


daydreamer

Recommended Posts

Hi.

 

I have written a web crawler script. It will visit a large number of URL's with cURL.

 

After around 2-3 minutes of running, it will just stop, with no error output or notices.

 

I have these settings:

set_time_limit(0);
ini_set('display_errors',1);
error_reporting(E_ALL|E_STRICT);

 

Any ideas why it would just stop?

If it's a long running task, it should almost certainly be run from the command line rather than from a we browser.

However, you do need to check the php.ini settings for the command line, it's quite possible that it's running against a different php.ini file. Running a phpinfo() script from the command line should provide that information

Archived

This topic is now archived and is closed to further replies.

×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.