Jump to content

PHP web crawler encoding


Beedy555

Recommended Posts

Hi,

 

i am using a php web crawler to extract adverts from various websites, these adverts are then displayed on my website however some anomalies are occurring. The encoding on the text is not correct, my Database is set to UTF_general_ci encoding, however my html files for my web are set to <meta http-

equiv="Content-Type" content="text/html; charset=iso-8859-1" />

 

i have attached two files to explain what i mean in more detail. Will i need to set UTF-8 encoding to all my files, as when i do this the characters do not appear to display properly

 

Thankyou

 

[attachment deleted by admin]

Link to comment
Share on other sites

From my experience..... you have to find what encoding the site you are connecting to uses, and then encode that to utf8 and save as utf8.

 

Is a few options.

http://php.net/manual/en/function.mb-detect-encoding.php

http://php.net/manual/ja/function.mb-convert-encoding.php

http://php.net/manual/en/book.iconv.php

 

Languages and encoding is one of the worst things the internet and server applications have not nailed out completely.

php6 was supposed to solve all this..but this has been in the shadows for a while now.

 

For any odd characters or your own special rules, you can make a function and do string replaces.

Browse the net, many have made similar functions already.

Link to comment
Share on other sites

thanks, i have changed the database collation to UTF-8 General, and included the correct tag in pages, but still get the error

 

Youll need to delete your existing fields and write them again. Also make sure the collation is "utf8_bin" not UTF-8 General

Link to comment
Share on other sites

This thread is more than a year old. Please don't revive it unless you have something important to add.

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.