Beedy555 Posted June 23, 2011 Share Posted June 23, 2011 Hi, i am using a php web crawler to extract adverts from various websites, these adverts are then displayed on my website however some anomalies are occurring. The encoding on the text is not correct, my Database is set to UTF_general_ci encoding, however my html files for my web are set to <meta http- equiv="Content-Type" content="text/html; charset=iso-8859-1" /> i have attached two files to explain what i mean in more detail. Will i need to set UTF-8 encoding to all my files, as when i do this the characters do not appear to display properly Thankyou [attachment deleted by admin] Link to comment https://forums.phpfreaks.com/topic/240185-php-web-crawler-encoding/ Share on other sites More sharing options...
dreamwest Posted June 23, 2011 Share Posted June 23, 2011 Change database collation to : Utf8_bin Change html header to: <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> Link to comment https://forums.phpfreaks.com/topic/240185-php-web-crawler-encoding/#findComment-1233742 Share on other sites More sharing options...
Beedy555 Posted June 23, 2011 Author Share Posted June 23, 2011 thanks, i have changed the database collation to UTF-8 General, and included the correct tag in pages, but still get the error Link to comment https://forums.phpfreaks.com/topic/240185-php-web-crawler-encoding/#findComment-1233788 Share on other sites More sharing options...
QuickOldCar Posted June 23, 2011 Share Posted June 23, 2011 From my experience..... you have to find what encoding the site you are connecting to uses, and then encode that to utf8 and save as utf8. Is a few options. http://php.net/manual/en/function.mb-detect-encoding.php http://php.net/manual/ja/function.mb-convert-encoding.php http://php.net/manual/en/book.iconv.php Languages and encoding is one of the worst things the internet and server applications have not nailed out completely. php6 was supposed to solve all this..but this has been in the shadows for a while now. For any odd characters or your own special rules, you can make a function and do string replaces. Browse the net, many have made similar functions already. Link to comment https://forums.phpfreaks.com/topic/240185-php-web-crawler-encoding/#findComment-1233809 Share on other sites More sharing options...
QuickOldCar Posted June 23, 2011 Share Posted June 23, 2011 If this helps you at all, I also add this before my mysql query to insure it gets saved as utf8. mysql_query("SET NAMES 'utf8'"); Link to comment https://forums.phpfreaks.com/topic/240185-php-web-crawler-encoding/#findComment-1233812 Share on other sites More sharing options...
dreamwest Posted June 23, 2011 Share Posted June 23, 2011 thanks, i have changed the database collation to UTF-8 General, and included the correct tag in pages, but still get the error Youll need to delete your existing fields and write them again. Also make sure the collation is "utf8_bin" not UTF-8 General Link to comment https://forums.phpfreaks.com/topic/240185-php-web-crawler-encoding/#findComment-1234084 Share on other sites More sharing options...
Recommended Posts
Archived
This topic is now archived and is closed to further replies.