FAQ
Hi,
I've searched the net but didn't find the information I need.
Using Python-2.7.1, I know, I can't modify defaultencoding at run time.
Python even ignores
export PYTHONIOENCODING=ISO8859-1

locale.getdefaultlocale()[1]
returns
'ISO8859-1'

still sys.stdout is using the ascii codec.
How can I recompile Python (itself) to change this to iso8859-1 ?
(My favourite editor cannot be told to use unicode.)

Many thanks for a hint,
Helmut.

Search Discussions

  • Helmut Jarausch at Jan 20, 2011 at 3:39 pm

    On Thu, 20 Jan 2011 14:31:09 +0000, Helmut Jarausch wrote:

    Hi,
    I've searched the net but didn't find the information I need. Using
    Python-2.7.1, I know, I can't modify defaultencoding at run time. Python
    even ignores
    export PYTHONIOENCODING=ISO8859-1

    locale.getdefaultlocale()[1]
    returns
    'ISO8859-1'

    still sys.stdout is using the ascii codec. How can I recompile Python
    (itself) to change this to iso8859-1 ? (My favourite editor cannot be
    told to use unicode.)
    Sorry for answering myself. One last trial did succeed.
    My locale as root differed from my locale as non-root user.
    After changing the root locale and recompiling Python, it works now.





    --
  • Philip Semanchuk at Jan 20, 2011 at 3:47 pm

    On Jan 20, 2011, at 10:39 AM, Helmut Jarausch wrote:
    On Thu, 20 Jan 2011 14:31:09 +0000, Helmut Jarausch wrote:

    Hi,
    I've searched the net but didn't find the information I need. Using
    Python-2.7.1, I know, I can't modify defaultencoding at run time. Python
    even ignores
    export PYTHONIOENCODING=ISO8859-1

    locale.getdefaultlocale()[1]
    returns
    'ISO8859-1'

    still sys.stdout is using the ascii codec. How can I recompile Python
    (itself) to change this to iso8859-1 ? (My favourite editor cannot be
    told to use unicode.)
    Sorry for answering myself. One last trial did succeed.
    My locale as root differed from my locale as non-root user.
    After changing the root locale and recompiling Python, it works now.
    I'm glad that worked for you. Alternatively, it seems like you can set the default encoding in site.py which sounds easier than recompiling Python.


    Cheers
    Philip
  • Robert Kern at Jan 20, 2011 at 4:47 pm

    On 1/20/11 9:47 AM, Philip Semanchuk wrote:

    I'm glad that worked for you. Alternatively, it seems like you can set the default encoding in site.py which sounds easier than recompiling Python.
    Never do that. It breaks dicts and makes your code non-portable.

    --
    Robert Kern

    "I have come to believe that the whole world is an enigma, a harmless enigma
    that is made terrible by our own mad attempt to interpret it as though it had
    an underlying truth."
    -- Umberto Eco
  • Philip Semanchuk at Jan 20, 2011 at 4:58 pm

    On Jan 20, 2011, at 11:47 AM, Robert Kern wrote:
    On 1/20/11 9:47 AM, Philip Semanchuk wrote:

    I'm glad that worked for you. Alternatively, it seems like you can set the default encoding in site.py which sounds easier than recompiling Python.
    Never do that. It breaks dicts and makes your code non-portable.
    I've never been tempted for the very non-portability reason you cite. I didn't know that it would break dicts, though. Thanks for the education.

    bye
    Philip
  • Robert Kern at Jan 20, 2011 at 4:46 pm

    On 1/20/11 8:31 AM, Helmut Jarausch wrote:
    Hi,
    I've searched the net but didn't find the information I need.
    Using Python-2.7.1, I know, I can't modify defaultencoding at run time.
    You're not supposed to. It must remain 'ascii'. Otherwise, you will break dict
    lookups among other things. Can you be specific about why you think you want to
    change this? What are you trying to achieve? It looks like you are conflating a
    variety of different behaviors below.
    Python even ignores
    export PYTHONIOENCODING=ISO8859-1

    locale.getdefaultlocale()[1]
    returns
    'ISO8859-1'
    This does not introspect the same thing as sys.getdefaultencoding().
    still sys.stdout is using the ascii codec.
    This reflects your terminal settings, not sys.getdefaultencoding(). I'm not sure
    why the PYTHONIOENCODING variable isn't affecting that. It works fine for me.
    How can I recompile Python (itself) to change this to iso8859-1 ?
    (My favourite editor cannot be told to use unicode.)
    You wouldn't want to change sys.getdefaultencoding() for this. That parameter
    affects how str and unicode objects are converted between each other without an
    explicit .encode()/.decode() call, not what encoding Python assumes for the
    parsing of the code.

    Instead, you want to use an encoding declaration in each file:

    http://docs.python.org/reference/lexical_analysis.html#encoding-declarations

    --
    Robert Kern

    "I have come to believe that the whole world is an enigma, a harmless enigma
    that is made terrible by our own mad attempt to interpret it as though it had
    an underlying truth."
    -- Umberto Eco
  • Helmut Jarausch at Jan 20, 2011 at 5:20 pm

    On Thu, 20 Jan 2011 10:46:37 -0600, Robert Kern wrote:
    On 1/20/11 8:31 AM, Helmut Jarausch wrote:
    Hi,
    I've searched the net but didn't find the information I need. Using
    Python-2.7.1, I know, I can't modify defaultencoding at run time.
    You're not supposed to. It must remain 'ascii'. Otherwise, you will
    break dict lookups among other things. Can you be specific about why you
    think you want to change this? What are you trying to achieve? It looks
    like you are conflating a variety of different behaviors below.
    Python even ignores
    export PYTHONIOENCODING=ISO8859-1

    locale.getdefaultlocale()[1]
    returns
    'ISO8859-1'
    This does not introspect the same thing as sys.getdefaultencoding().
    still sys.stdout is using the ascii codec.
    This reflects your terminal settings, not sys.getdefaultencoding(). I'm
    not sure why the PYTHONIOENCODING variable isn't affecting that. It
    works fine for me.
    How can I recompile Python (itself) to change this to iso8859-1 ? (My
    favourite editor cannot be told to use unicode.)
    You wouldn't want to change sys.getdefaultencoding() for this. That
    parameter affects how str and unicode objects are converted between each
    other without an explicit .encode()/.decode() call, not what encoding
    Python assumes for the parsing of the code.

    Instead, you want to use an encoding declaration in each file:

    http://docs.python.org/reference/lexical_analysis.html#encoding-
    declarations

    Thanks Robert,
    probably I wasn't too clear about my issue.
    I couldn't "print" any non-ascii character to my console although
    my console has an en_US.iso88591 locale.

    I didn't modfiy anything in Python's source code.
    I noticed that my root account still had an ASCII locale.
    Now I have changed it such that the root account has the same locale
    as non-root accounts. Recompiling Python under such a root account
    has rectified things. Now I can print non-ascii characters if they are
    properly encoded.

    Thanks,
    Helmut.




    --
  • Antoine Pitrou at Jan 20, 2011 at 7:37 pm

    On 20 Jan 2011 17:20:14 GMT Helmut Jarausch wrote:
    Thanks Robert,
    probably I wasn't too clear about my issue.
    I couldn't "print" any non-ascii character to my console although
    my console has an en_US.iso88591 locale.
    Well, if you want a correct answer, you should paste actual code as
    well as its result on your system, rather than start by asking how to
    change getdefaultencoding (which you shouldn't do as explained by
    others).
    I didn't modfiy anything in Python's source code.
    I noticed that my root account still had an ASCII locale.
    Now I have changed it such that the root account has the same locale
    as non-root accounts. Recompiling Python under such a root account
    has rectified things.
    Recompiling Python doesn't change its behaviour wrt. locales and
    encodings. All this is done at runtime.
    Now I can print non-ascii characters if they are
    properly encoded.
    You can *always* print characters if they are properly encoded. What
    you are asking is for Python to guess and do the encoding by itself,
    which is a different matter (and a poorly supported one under 2.x;
    Python 3 behaves much better in that regard).

    Regards

    Antoine.
  • John Pinner at Jan 21, 2011 at 11:43 am
    On Jan 20, 4:46?pm, Robert Kern wrote:

    <snips>
    Instead, you want to use an encoding declaration in each file:

    http://docs.python.org/reference/lexical_analysis.html#encoding-decla...
    All that this does is tell the interpreter how the source file is
    encoded, it does not affect default encodings etc.

    John
    --
  • Robert Kern at Jan 21, 2011 at 4:31 pm

    On 1/21/11 5:43 AM, John Pinner wrote:
    On Jan 20, 4:46 pm, Robert Kernwrote:

    <snips>
    Instead, you want to use an encoding declaration in each file:

    http://docs.python.org/reference/lexical_analysis.html#encoding-decla...
    All that this does is tell the interpreter how the source file is
    encoded, it does not affect default encodings etc.
    Yes! In the part of the OP's message that you snipped "(My favourite editor
    cannot be told to use unicode.)", that seemed to be part of his actual problem,
    not the default encoding.

    --
    Robert Kern

    "I have come to believe that the whole world is an enigma, a harmless enigma
    that is made terrible by our own mad attempt to interpret it as though it had
    an underlying truth."
    -- Umberto Eco
  • John Pinner at Jan 21, 2011 at 12:15 pm
    To answer the OP's original question:
    On Jan 20, 2:31?pm, Helmut Jarausch wrote:
    Hi,
    I've searched the net but didn't find the information I need.
    Using Python-2.7.1, I know, I can't modify defaultencoding at run time.
    I think you can. There is a function setdefaultencoding in the sys
    module, but at startup when site.py runs the function gets deleted
    after it has been used, because as others have said it is a *bad* idea
    to change the default encoding (although I *think* changing it to utf8
    should cause no harm).

    so if you reload sys, setdefaultencoding() becomes available again,
    and you can use it, with all the caveats mentioned:

    import sys
    reload(sys)
    sys.setdefaultencoding( 'utf-8' )

    This works on a Unicode build of Python only.

    As has been said, you can change the default encoding in site.py, but
    this means that it gets changed for everyone/every Python program on
    the system, using setdefaultencoding() as above only changes it for
    the running program, and hopefully the change will have been made by
    someone who knows what they are doing and is aware of the possible
    consequences.
    Python even ignores
    export PYTHONIOENCODING=ISO8859-1

    locale.getdefaultlocale()[1]
    returns
    'ISO8859-1'

    still sys.stdout is using the ascii codec.
    How can I recompile Python (itself) to change this to iso8859-1 ?
    You don't need to, nor should you.
    (My favourite editor cannot be told to use unicode.)
    Maybe you need a new editor. scite works well with Python, and is
    cross-platform.

    John
    --

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppython-list @
categoriespython
postedJan 20, '11 at 2:31p
activeJan 21, '11 at 4:31p
posts11
users5
websitepython.org

People

Translate

site design / logo © 2022 Grokbase