db323c Posted March 12, 2010 Share Posted March 12, 2010 Trying to create a script to grab all the threads from a forum and put them in a db to search through later. Everything seems to work except that I get a 500 internal server error after 5-10 minutes. set_time_limit(0); mysql_query("Truncate table forumindex"); //#of threads for($x=1; $x<40000; $x++) { //multiple forums for($i=0; $i<4; $i++) { //initial thought was a timeout issue, so I tried to solve that. $ctx = stream_context_create(array( ‘http’ => array( ‘timeout’ => 3 ) ) ); $handle = file_get_contents("http://*********.com/boardthread?id=" . $forum[$i] . "&thread=" . $x, null, $ctx); if($handle) { //code to add to db. which works. } } } Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.