jakebur01

<?php if (isset($_GET['limit_start'])) { $limit_start=$_GET['limit_start']; $limit_end=$_GET['limit_end']; } ini_set('max_execution_time', '999'); // example of how to use basic selector to retrieve HTML contents include('simple_html_dom.php'); $source_file = "C:/Inetpub/Websites/edit.com/echo.txt"; $fp= fopen("$source_file", "a"); $other="\n"; $db = mysql_connect('localhost', 'xx', 'xx') or die(mysql_error()); mysql_select_db('xx') or die(mysql_error()); $total_rows = mysql_num_rows(mysql_query("SELECT * FROM zip_code where `state_prefix` = 'OK'", $db)); $rows_per_loop = 25; $total_loops = ceil($total_rows/$rows_per_loop); // run the loop, while loop counter is less than the total number of loops: for($s=0; $s<$total_loops; $s++) { // get the numbers for the limit, // start is just the current loop number multiplied by the rows per loop // and end is loop counter + 1, multiplied by the rows per loop $limit_start = $rows_per_loop*$s; $limit_end = $rows_per_loop*($s+1); $result = mysql_query("SELECT * FROM zip_code where `state_prefix` = 'OK' LIMIT $limit_start, $limit_end") or die(mysql_error()); while($myrow = mysql_fetch_array ($result)) { $zip=$myrow['zip_code']; // get DOM from URL or file $html=file_get_html("http://www.www.edit.com/selector.php?transaction=search&template=map_search&search1=0&pwidth=400&pheight=700&proxIconId=400&proxIcons=1&search2=0&search3=1&country=US&searchQuantifier=AND&address=&city=&stateProvince=+&postalCode=$zip&radius=500&x=78&y=16"); $i = 0; $tmp = $html->find('span[class=mqEmp], span[class=Black11]'); $cnt = count($tmp) - 1; foreach($tmp as $e) { if($i > 0 && $i < $cnt){ $outputstring=$e->plaintext; $outputstring=$outputstring. "\t"; fwrite($fp, $outputstring, strlen($outputstring)); if ($i % 8 == 0) { fwrite($fp, $other, strlen($other)); } } $i++; } } // now that we've run those, let's clear the results so that we don't run out of memory. mysql_free_result($result); // show where we are at echo $limit_start,' - ',$limit_end,'<br />'; sleep(1); // give PHP a little nap to keep from overloading server ob_flush(); // as recommended by MadTechie, in case we go beyond the max execution time flush(); // add flush, to make sure it is outputted header( "location:http://www.edit.com/selector.php?limit_start=$limit_start&limit_end=$limit_end" ); } fclose($fp); ?>

I tried this, but it is not functioning right. It is looping through and sending multiple headers without executing it. I added this at the top. if (isset($_GET['limit_start'])) { $limit_start=$_GET['limit_start']; $limit_end=$_GET['limit_end']; } And this at the bottom // now that we've run those, let's clear the results so that we don't run out of memory. mysql_free_result($result); // show where we are at echo $limit_start,' - ',$limit_end,'<br />'; sleep(1); // give PHP a little nap to keep from overloading server ob_flush(); // as recommended by MadTechie, in case we go beyond the max execution time flush(); // add flush, to make sure it is outputted header( "location:http://www.mysite/example.php?limit_start=$limit_start&limit_end=$limit_end" ); } fclose($fp);

That's kind of what I already have going. Now I am trying to go to a new page after processing 25 rows and pick up where I left off using $_GET.

How could I redirect to myself and use Get to pass the last ID limiting 25 each time? example..... run 25 then redirect to: mypage.php?offset=25 run 25 more then redirect to mypage.php?offset=50 and so on.. $total_rows = mysql_num_rows(mysql_query("SELECT * FROM zip_code where `state_prefix` = 'OK'", $db)); $rows_per_loop = 20; $total_loops = ceil($total_rows/$rows_per_loop); // run the loop, while loop counter is less than the total number of loops: for($s=0; $s<$total_loops; $s++) { // get the numbers for the limit, // start is just the current loop number multiplied by the rows per loop // and end is loop counter + 1, multiplied by the rows per loop $limit_start = $rows_per_loop*$s; $limit_end = $rows_per_loop*($s+1); $result = mysql_query("SELECT * FROM zip_code where `state_prefix` = 'OK' LIMIT $limit_start, $limit_end") or die(mysql_error()); while($myrow = mysql_fetch_array ($result)) { $zip=$myrow['zip_code']; // get DOM from URL or file $html=file_get_html("http://www.mysite.com/age.asp?transaction=search&template=map_search&search1=0&pwidth=400&pheight=700&proxIconId=400&proxIcons=1&search2=0&search3=1&country=US&searchQuantifier=AND&address=&city=&stateProvince=+&postalCode=$zip&radius=500&x=78&y=16"); $i = 0; $tmp = $html->find('span[class=mqEmp], span[class=Black11]'); $cnt = count($tmp) - 1; foreach($tmp as $e) { if($i > 0 && $i < $cnt){ $outputstring=$e->plaintext; $outputstring=$outputstring. "\t"; fwrite($fp, $outputstring, strlen($outputstring)); if ($i % 8 == 0) { fwrite($fp, $other, strlen($other)); } } $i++; } } // now that we've run those, let's clear the results so that we don't run out of memory. mysql_free_result($result); // show where we are at echo $limit_start,' - ',$limit_end,'<br />'; sleep(1); // give PHP a little nap to keep from overloading server ob_flush(); // as recommended by MadTechie, in case we go beyond the max execution time flush(); // add flush, to make sure it is outputted } fclose($fp);

How could I make this like refresh to a new page limiting 25 each time? Like..... run 25 then redirect to: mypage.php?offset=25 run 25 more then redirect to mypage.php?offset=50 and so on..

bump

Is there any other way to give it a break in between?

I have tried that. I was referring to something that would maybe release the memory more often or something.

bump...

How can I free up the memory on this script to keep from getting " 0 - 20 20 - 40 PHP Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 1992 bytes) in ?" $total_rows = mysql_num_rows(mysql_query("SELECT * FROM zip_code where `state_prefix` = 'OK'", $db)); $rows_per_loop = 20; $total_loops = ceil($total_rows/$rows_per_loop); // run the loop, while loop counter is less than the total number of loops: for($s=0; $s<$total_loops; $s++) { // get the numbers for the limit, // start is just the current loop number multiplied by the rows per loop // and end is loop counter + 1, multiplied by the rows per loop $limit_start = $rows_per_loop*$s; $limit_end = $rows_per_loop*($s+1); $result = mysql_query("SELECT * FROM zip_code where `state_prefix` = 'OK' LIMIT $limit_start, $limit_end") or die(mysql_error()); while($myrow = mysql_fetch_array ($result)) { $zip=$myrow['zip_code']; // get DOM from URL or file $html=file_get_html("http://www.mysite.com/age.asp?transaction=search&template=map_search&search1=0&pwidth=400&pheight=700&proxIconId=400&proxIcons=1&search2=0&search3=1&country=US&searchQuantifier=AND&address=&city=&stateProvince=+&postalCode=$zip&radius=500&x=78&y=16"); $i = 0; $tmp = $html->find('span[class=mqEmp], span[class=Black11]'); $cnt = count($tmp) - 1; foreach($tmp as $e) { if($i > 0 && $i < $cnt){ $outputstring=$e->plaintext; $outputstring=$outputstring. "\t"; fwrite($fp, $outputstring, strlen($outputstring)); if ($i % 8 == 0) { fwrite($fp, $other, strlen($other)); } } $i++; } } // now that we've run those, let's clear the results so that we don't run out of memory. mysql_free_result($result); // show where we are at echo $limit_start,' - ',$limit_end,'<br />'; sleep(1); // give PHP a little nap to keep from overloading server ob_flush(); // as recommended by MadTechie, in case we go beyond the max execution time flush(); // add flush, to make sure it is outputted } fclose($fp);

Wow! That's exactly what I was looking for. Thanks a lot man.

Yea, but I need it to repeat after every eight rows.

I have searched and not been able to figure how to return to a new line after 8 number of loops or tabs when writing to a text file. I would like to write 8 fields then tab to a new line. Something like: $i = 0; $tmp = $html->find('span[class=mqEmp], span[class=Black11]'); $cnt = count($tmp) - 1; foreach($tmp as $e) { if($i > 0 && $i < $cnt){ $outputstring=$e->plaintext . '\t'; fwrite($fp, $outputstring, strlen($outputstring)); // if eighth field, then start a new row something like: fwrite($fp, \n); } $i++; }

It is not to bad if I limit it by one state at a time. It is actually faster than I expected after I fixed that little quote issue. I have search and not been able to figure how to tab after x number of loops when writing to a text file. I would like to write 8 fields then tab to a new line. Something like: $i = 0; $tmp = $html->find('span[class=mqEmp], span[class=Black11]'); $cnt = count($tmp) - 1; foreach($tmp as $e) { if($i > 0 && $i < $cnt){ $outputstring=$e->plaintext . '\t'; fwrite($fp, $outputstring, strlen($outputstring)); // if eighth field, then start a new row something like: fwrite($fp, \n); } $i++; }

I put double quotes instead of single quotes on file_get_html(), which corrected my $zip variable. But, it is still taking a few seconds even when I use LIMIT 1 in the query.

How can I optimize this to make it run better. As of now it will not even do a few zip codes. Is there any way to like sleep in between each loop or something? $result = mysql_query("SELECT * FROM zip_code where `state_prefix` = 'OK'", $db) or die(mysql_error()); while($myrow = mysql_fetch_array ($result)) { $zip=$myrow['zip_code']; $html = file_get_html('http://www.my-site.com/test.asp?transaction=search&template=map_search&search1=0&pwidth=400&pheight=700&proxIconId=400&proxIcons=1&search2=0&search3=1&country=US&searchQuantifier=AND&address=&city=&stateProvince=+&postalCode=$zip&radius=500&x=78&y=16'); $i = 0; $tmp = $html->find('span[class=mqEmp], span[class=Black11]'); $cnt = count($tmp) - 1; foreach($tmp as $e) { if($i > 0 && $i < $cnt){ echo $e->plaintext . '<br>'; } $i++; } }

Wait I think I got it. I added this in: $cnt = count($tmp); $cnt = $cnt-1;

For some reason it is not excluding the last row. However, it is excluding the first. - Jake

nevermind... Got it. THANKS!! $i = 0; $tmp = $html->find('span[class=mqEmp], span[class=Black11]'); $cnt = count($tmp); foreach($tmp as $e) { if($i > 0 && $i < $cnt){ echo $e->plaintext . '<br>'; } $i++; }

PHP Parse error: syntax error, unexpected '}'

How could I get this to exclude the first and last row? foreach($html->find('span[class=mqEmp], span[class=Black11]') as $e) echo $e->plaintext . '<br>'; Example: if the above returns... 112 test drive 99999 bob smith 384 california circle new park, CA 99999 john doe 384 california circle rockport, CA 94939 User assumes all risk of use. My Company, MapQuest and their suppliers assume no responsibility for any loss resulting from such use. How could I get rid of the first and last lines?

Would it be smart to use strtolower() when inserting a username into a database and also use it when comparing? if (isset($_POST['userid']) && isset($_POST['password'])) { // if the user has just tried to log in $userid = strtolower($_POST['userid']); $password = $_POST['password']; require("newlife_data.inc"); if (mysqli_connect_errno()) { echo 'Connection to database failed:'.mysqli_connect_error(); exit(); } $query = 'select * from life_useraccount ' ."where Username='$userid' " ." and Password=sha1('$password') " ." and Active='Y'"; $result = $db_conn->query($query); if ($result->num_rows >0 ) { }

Thanks Keith.

Man, you are amazing. It works perfect. Thank you. Where are some good places to learn more about MySQL queries?

Hi, I am trying to query the friend table (life_approval)columns(Id, UserId1, UserId2, and Approve) and see who all is a friend of the variable $q. I am trying to do this by checking both columns, if $q is a friend to someone..... he/she could be in either column. When a friend invite is stored in this table, the person the invite is sent to is stored in UserId1. {which my attempt appears to be pulling everyone out of the table in columns UserId1 and UserId2} The second query is supposed to be collecting information on the friend. I was trying to do that by using ( WHERE `Username` = 'UserId1 from the first query OR `Username` = 'UserId2' from the first query. Thanks, Jake