StefanRSA

Cool, I tried to use this formula given on php.net: function strip_word_html($text, $allowed_tags = '') { mb_regex_encoding('UTF-8'); //replace MS special characters first $search = array('/‘/u', '/’/u', '/“/u', '/”/u', '/—/u'); $replace = array('\'', '\'', '"', '"', '-'); $text = preg_replace($search, $replace, $text); //make sure _all_ html entities are converted to the plain ascii equivalents - it appears //in some MS headers, some html entities are encoded and some aren't $text = html_entity_decode($text, ENT_QUOTES, 'UTF-8'); //try to strip out any C style comments first, since these, embedded in html comments, seem to //prevent strip_tags from removing html comments (MS Word introduced combination) if(mb_stripos($text, '/*') !== FALSE){ $text = mb_eregi_replace('#/\*.*?\*/#s', '', $text, 'm'); } //introduce a space into any arithmetic expressions that could be caught by strip_tags so that they won't be //'<1' becomes '< 1'(note: somewhat application specific) $text = preg_replace(array('/<([0-9]+)/'), array('< $1'), $text); $text = strip_tags($text, $allowed_tags); //eliminate extraneous whitespace from start and end of line, or anywhere there are two or more spaces, convert it to one $text = preg_replace(array('/^\s\s+/', '/\s\s+$/', '/\s\s+/u'), array('', '', ' '), $text); //strip out inline css and simplify style tags $search = array('#<(strong|b)[^>]*>(.*?)</(strong|b)>#isu', '#<(em|i)[^>]*>(.*?)</(em|i)>#isu', '#<u[^>]*>(.*?)</u>#isu'); $replace = array('<b>$2</b>', '<i>$2</i>', '<u>$1</u>'); $text = preg_replace($search, $replace, $text); //on some of the ?newer MS Word exports, where you get conditionals of the form 'if gte mso 9', etc., it appears //that whatever is in one of the html comments prevents strip_tags from eradicating the html comment that contains //some MS Style Definitions - this last bit gets rid of any leftover comments */ $num_matches = preg_match_all("/\<!--/u", $text, $matches); if($num_matches){ $text = preg_replace('/\<!--(.)*--\>/isu', '', $text); } return $text; } I have two issues... 1. How light is this formula if I wanna parse an XML file of 2Gb (estimate 400 000 entries)... Will this have a bad impact on server load? 2. This also did not solve my problem with the "£;" issue that is still left by doing the folowing....: $fullt=strip_word_html($texta);/// $fullt=str_replace('£;',"£",$fullt); Will this then be safe to directly insert into a Db field? Thanks for all your help!!!

Thanks for the reply. How would i replace line breaks with proper validated line breaks? When using pure strip_tags it still leaves me with non text caracters like: ’ for ' – for - £; - for £

I am parsing an XML file and importing the data to Mysql Db. One of the nodes is text but contains random URL Links, sometimes css and html and sometimes multiple <br>, <lu>, <li> x2, x3 x4 I want to clean the text to be imported to be without URL links, css & html, but want to keep it tidy to be displayed as regular text on a webpage with regular spacing... I also need to get rid of untidy extra spaces and so more... The code I am using are not so fresh and give me broken text. What will be a more acceptable way than my following code? $fullt=str_replace("\n","",$texta); $fullt = preg_replace("/[\r\n]+/", "\n", $fullt); $fullt=trim($fullt); $fullt = preg_replace('|https?://www\.[a-z\.0-9]+|i', '', $fullt); $fullt = preg_replace('|www\.[a-z\.0-9]+|i', '', $fullt); $fullt=str_replace('">',"",$fullt); $fullt=str_replace('<"',"",$fullt); $fullt=preg_replace('/\s*$^\s*/m', "\n", $fullt); $fullt=str_replace('<BR/>',"<br>",$fullt); $fullt=str_replace('<br><br><br>',"<br>",$fullt); $fullt=str_replace('<br> <br>',"<br>",$fullt); $fullt=str_replace('&nbsp',"",$fullt); $fullt=str_replace('<br><br>',"<br>",$fullt); $fullt=str_replace('a href',"",$fullt); $fullt=str_replace('"',"",$fullt); $fullt=str_replace(';',"",$fullt); $fullt=str_replace(';',"",$fullt); $fullt=str_replace('^',"",$fullt); $qs='?'; $fullt = preg_replace( '@^(<br\\b[^>]*/'.$qs.'>)+@i', '', $fullt); //// Text ready $fullt=mysql_real_escape_string($fullt);

Why not doing a simple Lookup first to see if the userid exists and then do the Update or insert?

I need to import a 2Gb XML file into Mysql Db. The XML is built like the following sample <?xml version="1.0" encoding="UTF-8"?> <Properties> <Property> <ID></ID> <price></price> <price></price> <price></price> <price></price> <image id='1'> </image> <image id='2'> </image> <image id='3'> </image> </Property> <Property> ....... </Property> <Properties> I am now trying to get the id, the value of the 4th price node as well as the two first image node values... I am stuck... Please help. My script sofar: <? $reader = new XMLReader(); $reader->open($url); while($reader->read()){ if($reader->nodeType == XMLReader::ELEMENT) $nodeName = $reader->name; if($reader->nodeType == XMLReader::TEXT || $reader->nodeType == XMLReader::CDATA) { if ($nodeName == 'ID'){ $id = $reader->value; echo $id; } if ($nodeName == 'Price'){ $price = $reader->value; echo $price['4']; } if ($nodeName == 'Image'){ if($reader->getAttribute("id") == '1') { $image1= $reader->value; }else if($reader->getAttribute("id") == '2'){ $image2= $reader->value; } } } } ?>

I am trying the XMLReader as follow: (This is not working... Any help?) #!/usr/bin/php –q <? error_reporting(E_ALL); ini_set('display_errors', '1'); $url='http://xxxxxxx.xxx'; $reader = new XMLReader(); $reader->open($url); while($reader->read()){ if($reader->nodeType == XMLReader::ELEMENT) $nodeName = $reader->name; if($reader->nodeType == XMLReader::TEXT || $reader->nodeType == XMLReader::CDATA) { if ($nodeName == 'ID'){ $adnr = $reader->value; echo $adnr.'<br>'; } if ($nodeName == 'Country'){ $country= $reader->value; } if ($nodeName == 'Location1'){ $province = $reader->value; } if ($nodeName == 'Location2'){ $region = $reader->value; } if ($nodeName == 'Location3'){ $town = $reader->value; } if ($nodeName == 'Price'){ $price =$reader->getAttribute('4'); echo 'Price='.$price.'/n/n'; } if ($nodeName == 'Beds'){ $bedrooms = $reader->value; } if ($nodeName == 'Ref'){ $ref = $reader->value; } if ($nodeName == 'Name'){ $bname = $reader->value; } if ($nodeName == 'Sector'){ $sector = $reader->value; $sector=trim(preg_replace('/\s+/', ' ', $sector)); } if ($nodeName == 'Type'){ $type = $reader->value; $hometype = trim($type.' '.$sector); $hometype=trim(preg_replace('/\s+/', ' ', $hometype)); } if ($nodeName == 'Key_selling_points'){ $keyfield = $reader->value; } if ($nodeName == 'Baths'){ $bathrooms = $reader->value; } if ($nodeName == 'Full_Desc'){ $fullt = $reader->value; } if ($nodeName == 'image'){ $pic1 = $reader->getAttribute('1')->value; } if ($nodeName == 'image'){ $pic2 = $reader->getAttribute('2')->value; } if ($nodeName == 'image'){ $pic3 = $reader->getAttribute('3')->value; } if ($nodeName == 'image'){ $pic4 = $reader->getAttribute('4')->value; } if ($nodeName == 'image'){ $pic5 = $reader->getAttribute('5')->value; } if ($nodeName == 'image'){ $pic6 = $reader->getAttribute('6')->value; } ///////////////////////////////////////// } $count='0'; if($reader->nodeType == XMLReader::END_ELEMENT && $reader->name == 'Property') { $count++; } } $reader->close(); echo $count; ?>

I need to import a large XML file into my DB (2Gb) I tried to use DOMDocument and it failed cause of size and read up and understand that I should rather use XMLReader. Google search do not help much to give me an indication what I should different to read the document with XMLReader... Can anybody please help me to either point me in the right direction to fix this document I created or even help me to rewrite the parser... I just cannot get my head arounf this problem... My DOMDocument is as follow: #!/usr/bin/php –q <? error_reporting(E_ALL); ini_set('display_errors', '1'); $l = mysql_connect ( XXXXX); $total='0'; $xml = "http://xxxxxxxxxxxxxxxxxxxx"; $doc = new DOMDocument(); $doc->load( $xml); ## lets read the code block $records = $doc->getElementsByTagName( "Property" ); foreach( $records as $result ) { $adnr = $result->getElementsByTagName( "ID" ); // adnr $adnr = $adnr->item(0)->nodeValue; $town = $town->getElementsByTagName( "town" ); // town $town = $town->item(0)->nodeValue; //////////////////////////////////////////////////////////////////////////////// Add Pics $pic1= $result->getElementsByTagName( "image" ); $pic1 = $pic1->item('1')->nodeValue; $pic2= $result->getElementsByTagName( "image" ); $pic2 = $pic2->item('2')->nodeValue; $pic3= $result->getElementsByTagName( "image" ); $pic3 = $pic3->item('3')->nodeValue; } ?>

//$name='newpicfile'; $url=$imgurl; // create the context set_time_limit(-1); $img = file_get_contents($url); //////// GET IMAGE NAME $url_arr = explode ('/', $url); $ct = count($url_arr); $name = $url_arr[$ct-1]; $name_div = explode('.', $name); $ct_dot = count($name_div); $img_type = $name_div[$ct_dot -1]; $name=$adnr.'_'.$inr.'_'.$name; /////// GET IMAGE NAME END $im = imagecreatefromstring($img); $width = imagesx($im); $height = imagesy($im); $newwidth = '400'; $newheight = '240'; $thumb = imagecreatetruecolor($newwidth, $newheight); imagecopyresized($thumb, $im, 0, 0, 0, 0, $newwidth, $newheight, $width, $height); imagejpeg($thumb,'../ad_images/'.$name); //save image as jpg //echo '<img src="../ad_images/'.$name.'"> '; $newwidth1 = '130'; $newheight1 = '75'; $thumb1 = imagecreatetruecolor($newwidth1, $newheight1); imagecopyresized($thumb1, $im, 0, 0, 0, 0, $newwidth1, $newheight1, $width, $height); imagejpeg($thumb1,'../ad_images/130/'.$name); //save image as jpg //echo '<img src="../ad_images/130/'.$name.'"> '; $newwidth2 = '65'; $newheight2 = '37'; $thumb2 = imagecreatetruecolor($newwidth2, $newheight2); imagecopyresized($thumb2, $im, 0, 0, 0, 0, $newwidth2, $newheight2, $width, $height); imagejpeg($thumb2,'../ad_images/65/'.$name); //save image as jpg echo '<img src="../ad_images/65/'.$name.'"> <br>'; $dbname=$name; $pquery = "INSERT INTO *****"; mysql_query($pquery); imagedestroy($thumb); imagedestroy($thumb1); imagedestroy($thumb2); imagedestroy($im); Am sure this will help somewhere, someone else... ;-) Thanks for the reply mac

Hi, I am getting an image URL from an xml feed. I am trying to copy the image from the URL in the XML Feed, then rename & resize it. Its only giving me a black resized block for each image resize... This is driving me nuts. Please look at my script and tell me where am I going wrong? set_time_limit(-1); $imgurl='http://path/to/img_main.jpg'; $pic=$imgurl; $img = $pic; //////// GET IMAGE NAME $url_arr = explode ('/', $pic); $ct = count($url_arr); $name = $url_arr[$ct-1]; $name_div = explode('.', $name); $ct_dot = count($name_div); $img_type = $name_div[$ct_dot -1]; /////// GET IMAGE NAME END $name= str_replace(' ', '_', $name); $im = imagecreatefromjpeg($img); $width = imagesx($im); $height = imagesy($im); $newwidth = '400'; $newheight = '240'; $thumb = imagecreatetruecolor($newwidth, $newheight); imagecopyresized($thumb, $im, 0, 0, 0, 0, $newwidth, $newheight, $width, $height); imagejpeg($thumb,'../ad_images/'.$name); //save image as jpg echo '<img src="../ad_images/'.$name.'"> '; $newwidth1 = '130'; $newheight1 = '98'; $thumb1 = imagecreatetruecolor($newwidth1, $newheight1); imagecopyresized($thumb1, $im, 0, 0, 0, 0, $newwidth1, $newheight1, $width, $height); imagejpeg($thumb1,'../ad_images/130/'.$name); //save image as jpg echo '<img src="../ad_images/130/'.$name.'"> '; $newwidth2 = '65'; $newheight2 = '49'; $thumb2 = imagecreatetruecolor($newwidth2, $newheight2); imagecopyresized($thumb2, $im, 0, 0, 0, 0, $newwidth2, $newheight2, $width, $height); imagejpeg($thumb2,'../ad_images/65/'.$name); //save image as jpg echo '<img src="../ad_images/65/'.$name.'"> <br>'; $dbname=$name; imagedestroy($thumb); imagedestroy($thumb1); imagedestroy($thumb2); imagedestroy($im);

Thanks Lemmin... Was never thinking to do it straight on the MySql query. Will test it, thanks

I have an action that sets current time in the DB. I want to run a cronjob on a 5min interval to check for documents that is not set 5min ahead of time when cron runs, but also not in the past of when the cron runs and then perform a certain action.... Id tried it with the following code but its not working... Any ideas? $t=date("H:i:s", time()); echo $t.'<br>'; $currentTime = strtotime($t); $futureTime = $currentTime+(60*5); $plus5min = date("H:i:s", $futureTime); echo $plus5min; $inbetween = $currentTime+(60*3); $plus3min = date("H:i:s", $inbetween); echo '<br><br>'.$plus3min; if (($plus3min<=$plus5min) && ($plus3min>=$currentTime)){ echo '<br><br>inbetween'; }else{ echo '<br><br>Not Now'; }

Thanks for the help... My join order was wrong...

Sorry Barand, Yes, the "sct" column is indexed.

timcadieux - I just tried version one and did not really see an improvement... Will try V2 Barand - Yes, all related columns are indexed...

I have a DB with 400 000+ entries and am trying to split it into 10 entries per page... My usual paging for smaller DB rows are working fine with no loading issues. Just with the large table (with joins) it bacomes a very long page load. I am not sure what should be done to improve page load.... The script I have for paging works as follow: $query2 = mysql_query("SELECT COUNT( * ) FROM table_in WHERE table_in.scat='$blcid' $wprov $wtown"); list($nrResults) = mysql_fetch_row($query2); //$nrResults=mysql_num_rows($result2); IF (($nrResults%$limit)<>0) { $pmax=floor($nrResults/$limit)+1; // Divide to total result by the number of rows needed per page // to display per page($limit) and create a Max page } ELSE { $pmax=floor($nrResults/$limit); } $pagenumber=(($pages-1)*$limit); $query= "SELECT field, field, field, field FROM table_in JOIN s_tow ON s_tow.tId=table_in.tow JOIN s_reg ON s_reg.rId=table_in.ar JOIN s_pro ON s_pro.pId=table_in.pro JOIN bt ON bt.id=table_in.lid WHERE table_in.sct='$blcid' $wprov $wtown GROUP BY table.lid ORDER BY date $nav limit $pagenumber, $limit"; $allads=mysql_query($query) or die(mysql_error()); This works fine if I have up to 500 rows.... But cause havoc with 400 000 rows.... What should I do?