FAQ
I'm sure I had this working before but it's all gone very strange now.

I have a MySQL db where all the tables contain UTF8 data. Actually, as
far as I can see, the only problematic character is the ? sign.

From my understanding of MySQL, I think this select proves that the
data is UTF8:

mysql> select ord(substring(description, 48, 1)) from invoice_line where
invoice = 347;
+------------------------------------+
ord(substring(description, 48, 1)) |
+------------------------------------+
49827 |
49827 |
49827 |
49827 |
49827 |
+------------------------------------+
5 rows in set (0.03 sec)

49827 is (194 * 256) + 163 - hence 0xC2 0xA3[1].

My DBIC connect looks like this:

my $sch = MyData::DB->connect(
'dbi:mysql:database=mydata;',
'user', 'xxxx', { mysql_enable_utf8 => 1 }
);

But when I look at the data in the description attribute within the
Perl, the ? sign is just 0xA3 - which is the Latin1 encoding. The 0xC2
has been dropped.

DB<2> x map { ord $_ } (split //, $_->description)[45 .. 50]
0 64
1 32
2 163
3 52
4 48
5 48

(Those characters in the string are '@ ?400')

If I run:

$_->description(encode('utf8', decode('latin1', $_->description)));

Then I get:

DB<4> x map { ord $_ } (split //, $_->description)[45 .. 51]
0 64
1 32
2 194
3 163
4 52
5 48
6 48

Which is what I was expecting in the first place.

I thought that { mysql_enable_utf8 => 1 } was supposed to be the
solution to all these problems. But I'm obviously missing something.

This is one of those occasions where I'm hoping that I'm being really
stupid and there's an obvious problem staring me in the face.

Can you see it? :-/

Cheers,

Dave...

[1] See http://www.fileformat.info/info/unicode/char/a3/index.htm

Search Discussions

  • Peter Edwards at Jul 3, 2011 at 11:31 am

    On 3 July 2011 12:02, Dave Cross wrote:
    I have a MySQL db where all the tables contain UTF8 data. Actually, as far
    as I can see, the only problematic character is the £ sign.

    My DBIC connect looks like this:

    my $sch = MyData::DB->connect(
    'dbi:mysql:database=mydata;',
    'user', 'xxxx', { mysql_enable_utf8 => 1 }
    );

    Looking at
    https://github.com/dbsrgits/dbix-class/blob/master/lib/DBIx/Class/Storage/DBI/mysql.pm

    it doesn't look like it does a SET NAMES utf8; on initial connection and
    you're not using an on_connect_do to set it so unless your mysql db conf is
    set to default you might get issues?
    Maybe worth using strict mode too :-\

    Regards, Peter
    http://perl.dragonstaff.co.uk



    http://dev.mysql.com/doc/refman/5.0/en/charset-unicode.html
    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: http://lists.scsys.co.uk/pipermail/dbix-class/attachments/20110703/6acf12a7/attachment.htm

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupdbix-class @
categoriesperl, catalyst
postedJul 3, '11 at 11:01a
activeJul 3, '11 at 11:31a
posts2
users2
websitedbix-class.org
irc#dbix-class

2 users in discussion

Peter Edwards: 1 post Dave Cross: 1 post

People

Translate

site design / logo © 2021 Grokbase