After upgrading to pg 9.0.3 (from 8.4.2) on my Mac OS 10.6.2 machine i find this in my log file (a lot):

<postgres%192.168.254.210%2011-03-03 16:37:30 CET%22021>STATEMENT: SELECT pg_file_read('pg_log/postgresql-2011-03-03_000000.log', 250000, $
<postgres%192.168.254.210%2011-03-03 16:37:32 CET%22021>ERROR: invalid byte sequence for encoding "UTF8": 0xe3bc74

Apparently pg doesn't like the contents of that logfile.
The folks from the pgadmin list (i noticed the problem using the pgadmin logviewer and asked for help there) advised me to change LC_messages locale to 'C' (i had it on 'de_DE-UTF8' before), but that doesn't appear to help. The Server encoding is UTF8. No special client encoding is set.

Any help would be appreciated,

Max

Maximilian Tyrtania Software-Entwicklung
Dessauer Str. 6-7
10969 Berlin
http://www.contactking.de

Search Discussions

  • Craig Ringer at Mar 4, 2011 at 10:01 am

    On 04/03/11 00:02, Maximilian Tyrtania wrote:
    After upgrading to pg 9.0.3 (from 8.4.2) on my Mac OS 10.6.2 machine i find this in my log file (a lot):

    <postgres%192.168.254.210%2011-03-03 16:37:30 CET%22021>STATEMENT: SELECT pg_file_read('pg_log/postgresql-2011-03-03_000000.log', 250000, $
    <postgres%192.168.254.210%2011-03-03 16:37:32 CET%22021>ERROR: invalid byte sequence for encoding "UTF8": 0xe3bc74
    The "0xe3bc74" looks like gibberish in any encoding I can think of.
    What's the input file? Is it sanely encoded? Do you know what encoding
    it is in?

    If you really want to be encoding-agnostic and you do not care if you
    get garbage data in your database that makes no sense and can never make
    any sense, then you must ensure that your database is in the "C" locale
    for LC_CTYPE and LC_COLLATE, and you must SET client_encoding =
    "SQL_ASCII" when reading the data.

    A suitable CREATE DATABASE command might be:

    CREATE DATABASE garbage
    TEMPLATE template0
    ENCODING 'SQL_ASCII' LC_COLLATE 'C' LC_CTYPE 'C';

    but I really don't think that's generally a good idea. Storing random
    crap in text fields will cause you pain later. Better to either convert
    the text to a sane encoding, store it as bytea if you want the raw
    bytes, or reject it.

    --
    Craig Ringer
  • Maximilian Tyrtania at Mar 4, 2011 at 2:19 pm

    Am 04.03.2011 um 11:01 schrieb Craig Ringer:
    On 04/03/11 00:02, Maximilian Tyrtania wrote:
    After upgrading to pg 9.0.3 (from 8.4.2) on my Mac OS 10.6.2 machine i find this in my log file (a lot):

    <postgres%192.168.254.210%2011-03-03 16:37:30 CET%22021>STATEMENT: SELECT pg_file_read('pg_log/postgresql-2011-03-03_000000.log', 250000, $
    <postgres%192.168.254.210%2011-03-03 16:37:32 CET%22021>ERROR: invalid byte sequence for encoding "UTF8": 0xe3bc74
    The "0xe3bc74" looks like gibberish in any encoding I can think of.
    What's the input file?
    We are talking about pg's own logfile here. I thought that was clear. Look at the file's name. Apparently some guy on the french pgAdmin list has the very same problem. I have no idea how "0xe3bc74" made it into the log file.
    Is it sanely encoded? Do you know what encoding
    it is in?
    As i said, i initially set lc_messages to 'de_DE-UTF8', so i assume that's what the log file was in. I changed it to 'c' now.
    If you really want to be encoding-agnostic and you do not care if you
    get garbage data in your database that makes no sense and can never make
    any sense, then you must ensure that your database is in the "C" locale
    for LC_CTYPE and LC_COLLATE, and you must SET client_encoding =
    "SQL_ASCII" when reading the data.

    A suitable CREATE DATABASE command might be:

    CREATE DATABASE garbage
    TEMPLATE template0
    ENCODING 'SQL_ASCII' LC_COLLATE 'C' LC_CTYPE 'C';

    but I really don't think that's generally a good idea. Storing random
    crap in text fields will cause you pain later. Better to either convert
    the text to a sane encoding, store it as bytea if you want the raw
    bytes, or reject it.
    I certainly don't want to be encoding agnostic. I just would like to be able to read my log file using PGAdmin, which i can't right now, because PGAdmin 1.12. chops off the content after the 1st character that doesn't match the encoding.

    Best wishes,
    Max

    Maximilian Tyrtania Software-Entwicklung
    Dessauer Str. 6-7
    10969 Berlin
    http://www.contactking.de
  • Craig Ringer at Mar 5, 2011 at 6:52 am

    On 4/03/2011 10:18 PM, Maximilian Tyrtania wrote:
    Am 04.03.2011 um 11:01 schrieb Craig Ringer:
    On 04/03/11 00:02, Maximilian Tyrtania wrote:
    After upgrading to pg 9.0.3 (from 8.4.2) on my Mac OS 10.6.2 machine i find this in my log file (a lot):

    <postgres%192.168.254.210%2011-03-03 16:37:30 CET%22021>STATEMENT: SELECT pg_file_read('pg_log/postgresql-2011-03-03_000000.log', 250000, $
    <postgres%192.168.254.210%2011-03-03 16:37:32 CET%22021>ERROR: invalid byte sequence for encoding "UTF8": 0xe3bc74
    The "0xe3bc74" looks like gibberish in any encoding I can think of.
    What's the input file?
    We are talking about pg's own logfile here. I thought that was clear. Look at the file's name. Apparently some guy on the french pgAdmin list has the very same problem. I have no idea how "0xe3bc74" made it into the log file.
    Oh. Good point.

    PostgreSQL's logs suffer from an interesting bug in situations where the
    server uses mixed encodings, causing the log files to contain text with
    more than one encoding. There's been prior discussion of it, but no
    conclusions.
    Is it sanely encoded? Do you know what encoding
    it is in?
    As i said, i initially set lc_messages to 'de_DE-UTF8', so i assume that's what the log file was in. I changed it to 'c' now.
    Well, the PostgreSQL log files are *not* sanely encoded, so that's part
    of the problem.

    --
    Craig Ringer

    Tech-related writing at http://soapyfrogs.blogspot.com/

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppgsql-general @
categoriespostgresql
postedMar 3, '11 at 4:02p
activeMar 5, '11 at 6:52a
posts4
users2
websitepostgresql.org
irc#postgresql

People

Translate

site design / logo © 2022 Grokbase