Weird UTF-8 problem

Tanax · 06-19-2009, 04:07 PM

Hey!

I've hit a weird error.
åäö gets all weird signs(like squares with a ? in it), but all the charset is set to utf-8 so it should work.

All the files are encoded as utf-8 without BOM.

It worked great(without the weird signs) before I added some stuff to the database, and now those gets weird signs in it.
I checked my database, and it's the way it's supposed to be.. columns are swedish_latin_ci or w.e it's called.. :S

Anyone have an idea?

Wildhoney · 06-19-2009, 04:33 PM

Try setting your database collation to utf8_general_ci, and that the output in the browser is UTF-8 by viewing the page information.

You can also try running the following query:

sql Code:

SET NAMES 'utf8'

codefreek · 06-19-2009, 05:10 PM

You can also skip setting a language preset in the database it works as well :)

EDIT:
also you can run:
html_entities on it, that works & nl2br

OR

Set,

ISO Latin 1

AND:
set this,
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />

Tanax · 06-19-2009, 05:17 PM

I ran that query.
I set all the columns to utf8_general_ci.
I set the table to utf8_general_ci.
And the database was already utf8_general_ci.

None of it worked..
Could it be something in the connection on the PHP side?
'Cause all the rest of the text that is staticly printed on the page is working perfectly.

Wildhoney · 06-19-2009, 05:23 PM

Is the data being output to the browser in UTF-8?

Note Encoding.

http://www.grabup.com/uploads/600019...1809237F10.png

codefreek · 06-19-2009, 05:24 PM

Read my post..

Tanax · 06-19-2009, 07:42 PM

@codefreek- Charset on the html page is already utf8, and "SET Latin1"... would only bring it back to how it was in the beginning because that's what it was at first. html_entities wouldn't work, and nl2br shouldn't work either as it only converts nls(\n - ENTER) to <br />.

@wildhoney-
It looks like this:
http://img34.imageshack.us/img34/7146/utferror.jpg

Btw, as you see.. the staticly printed "Användarpanelen" at top right is output correctly, so there's nothing wrong with the file-encoding. There must be something with the database since it's only the data from the database that are being shown with those errors..

Wildhoney · 06-19-2009, 11:01 PM

A simple test and it appears to work for me, even with the page encoding as ISO-8859-1.

php Code:

$szSQL = "SELECT * FROM swedish";
$pResultSet = mysql_query($szSQL);
$pResult = mysql_fetch_row($pResultSet);

echo $pResult[1];

That code on a blank canvas with the table as so: http://www.grabup.com/uploads/430020...0009007F40.png

codefreek · 06-20-2009, 01:19 AM

Tanax, well hum i did all those thing's before and it worked for me but you might wanna google on it :)

codefreek · 06-20-2009, 01:40 AM

Wildhoney, what Database editor are you using, in this pic..
http://www.grabup.com/uploads/430020...0009007F40.png

Wildhoney · 06-20-2009, 02:51 AM

I use SQLYog

Tanax · 06-20-2009, 01:10 PM

Argh, so how come it doesn't work for me

It should work!!

sketchMedia · 06-20-2009, 07:05 PM

You could try sending the correct HTTP header (if your not already :) )

PHP Code:


			
header('Content-type: text/html; charset=UTF-8');

Tanax · 06-21-2009, 12:06 AM

Well, by looking at the pageinfo and the charset of the output on the page, it's set to UTF-8 already..

Could this be something due to I'm using a MVC-framework? I remember I had some troubles with this while using CodeIgniter..
I'm using my own MVC-framework right now, but could probably be the same issue here. Though I have no idea how to solve it since I don't have the options that I changed in order to make CI work correctly..

Could that be the problem anyways?

codefreek · 06-21-2009, 03:09 AM

tanax, can be but, should not be the problem..

sketchMedia · 06-22-2009, 10:36 AM

you could try converting the contents of your DB to utf with iconv.

NOTE:
the text 'sketch@talkphp:~$' is my attempt at emulating the command line

Code:

sketch@talkphp:~$ mysqldump -u user -p databasename > databasename.sql

You may want to create a backup DB:

Code:

sketch@talkphp:~$ mysql -u username -p
mysql>create database databasename-backup;
mysql>exit
Bye
sketch@talkphp:~$ mysql -u username -p databasename-backup < databasename.sql

Now we can run the dump through iconv

Code:

sketch@talkphp:~$ iconv -f Latin1 -t utf8 databasename.sql > databasename-iconv.sql

Now create a new DB or reuse the old on and dump it back in (making sure that the tables and database are set to utf8):

Code:

sketch@talkphp:~$ mysql -u root -p databasename < databasename-iconv.sql

Sorry, i don't know how you would do this on redmond.

Alternitively you could try this fudge.
basically convert the column with the problem data in to a binary format then convert back the charset to utf8 again, this should force mysql to convert the data.

Code:

sketch@talkphp:~$ mysql -u root -p databasename
mysql>alter table tbl_name modify column col_name CHAR(255) CHARACTER SET binary;
mysql> alter table tbl_name modify column col_name CHAR(255) CHARACTER SET utf8;

obviously replace the 'CHAR(255)' with whatever the current data type is.

I haven't tested it, so you would be best advised to create a backup!

Tanax · 06-22-2009, 08:48 PM

Wow, that is waay too complexed for me

Damnit. I've had åäö working with databases before :S

It must be because of my framework..

codefreek · 06-23-2009, 09:36 AM

Tanax: try without your framework ?