FAQ
hello,

Moving my entire program section between windows and Ubuntu,
sometimes causes problems, due to the existence of pyc-files
(and probably because my program still has hard coded paths).

Now I want get rid of the pyc-files,
so I wrote a py-script to remoce all pyc-files,
but because it's run from the same program section,
a few pyc files are recreated.

Is there a way to prevent generating pyc-files ?
Or is there a way to redirect the generated pyc-files to a dedicated
location ?

btw, What commandline switches are available for python ?
(googling didn't give me any relevant hits )

thanks,
Stef Mientki

Search Discussions

  • Mike Kazantsev at May 24, 2009 at 1:07 pm

    On Sun, 24 May 2009 15:01:51 +0200 Stef Mientki wrote:

    Moving my entire program section between windows and Ubuntu,
    sometimes causes problems, due to the existence of pyc-files
    (and probably because my program still has hard coded paths).

    Now I want get rid of the pyc-files,
    so I wrote a py-script to remoce all pyc-files,
    but because it's run from the same program section,
    a few pyc files are recreated.

    Is there a way to prevent generating pyc-files ?
    Or is there a way to redirect the generated pyc-files to a dedicated
    location ?
    Use a "-B" command-line option or "PYTHONDONTWRITEBYTECODE=x" env var.
    You can put either "alias python='python -B'" or
    "export PYTHONDONTWRITEBYTECODE=x" to your .bashrc/profile and forget
    about .pyc/pyo forever.
    btw, What commandline switches are available for python ?
    (googling didn't give me any relevant hits )
    You might be amazed how much insight "man python" and "python -h" can
    yield ;)

    --
    Mike Kazantsev // fraggod.net
    -------------- next part --------------
    A non-text attachment was scrubbed...
    Name: signature.asc
    Type: application/pgp-signature
    Size: 198 bytes
    Desc: not available
    URL: <http://mail.python.org/pipermail/python-list/attachments/20090524/d0a25c2e/attachment.pgp>
  • Esmail at May 24, 2009 at 1:15 pm

    Stef Mientki wrote:
    btw, What commandline switches are available for python ?
    (googling didn't give me any relevant hits )
    Hi Stef,

    This is what I get w/ v2.6 under Ubuntu 9.04


    9:12 esmail at t61 ~/Python [510] python -h
    usage: python [option] ... [-c cmd | -m mod | file | -] [arg] ...
    Options and arguments (and corresponding environment variables):
    -B : don't write .py[co] files on import; also PYTHONDONTWRITEBYTECODE=x
    -c cmd : program passed in as string (terminates option list)
    -d : debug output from parser; also PYTHONDEBUG=x
    -E : ignore PYTHON* environment variables (such as PYTHONPATH)
    -h : print this help message and exit (also --help)
    -i : inspect interactively after running script; forces a prompt even
    if stdin does not appear to be a terminal; also PYTHONINSPECT=x
    -m mod : run library module as a script (terminates option list)
    -O : optimize generated bytecode slightly; also PYTHONOPTIMIZE=x
    -OO : remove doc-strings in addition to the -O optimizations
    -Q arg : division options: -Qold (default), -Qwarn, -Qwarnall, -Qnew
    -s : don't add user site directory to sys.path; also PYTHONNOUSERSITE
    -S : don't imply 'import site' on initialization
    -t : issue warnings about inconsistent tab usage (-tt: issue errors)
    -u : unbuffered binary stdout and stderr; also PYTHONUNBUFFERED=x
    see man page for details on internal buffering relating to '-u'
    -v : verbose (trace import statements); also PYTHONVERBOSE=x
    can be supplied multiple times to increase verbosity
    -V : print the Python version number and exit (also --version)
    -W arg : warning control; arg is action:message:category:module:lineno
    -x : skip first line of source, allowing use of non-Unix forms of #!cmd
    -3 : warn about Python 3.x incompatibilities that 2to3 cannot trivially fix
    file : program read from script file
    - : program read from stdin (default; interactive mode if a tty)
    arg ...: arguments passed to program in sys.argv[1:]

    Other environment variables:
    PYTHONSTARTUP: file executed on interactive startup (no default)
    PYTHONPATH : ':'-separated list of directories prefixed to the
    default module search path. The result is sys.path.
    PYTHONHOME : alternate <prefix> directory (or <prefix>:<exec_prefix>).
    The default module search path uses <prefix>/pythonX.X.
    PYTHONCASEOK : ignore case in 'import' statements (Windows).
    PYTHONIOENCODING: Encoding[:errors] used for stdin/stdout/stderr.

    XP looks similar enough (I didn't look through it for any differences):

    Microsoft Windows XP [Version 5.1.2600]
    (C) Copyright 1985-2001 Microsoft Corp.

    C:\> python -h
    usage: python [option] ... [-c cmd | -m mod | file | -] [arg] ...
    Options and arguments (and corresponding environment variables):
    -B : don't write .py[co] files on import; also PYTHONDONTWRITEBYTECODE=x
    -c cmd : program passed in as string (terminates option list)
    -d : debug output from parser; also PYTHONDEBUG=x
    -E : ignore PYTHON* environment variables (such as PYTHONPATH)
    -h : print this help message and exit (also --help)
    -i : inspect interactively after running script; forces a prompt even
    if stdin does not appear to be a terminal; also PYTHONINSPECT=x
    -m mod : run library module as a script (terminates option list)
    -O : optimize generated bytecode slightly; also PYTHONOPTIMIZE=x
    -OO : remove doc-strings in addition to the -O optimizations
    -Q arg : division options: -Qold (default), -Qwarn, -Qwarnall, -Qnew
    -s : don't add user site directory to sys.path; also PYTHONNOUSERSITE
    -S : don't imply 'import site' on initialization
    -t : issue warnings about inconsistent tab usage (-tt: issue errors)
    -u : unbuffered binary stdout and stderr; also PYTHONUNBUFFERED=x
    see man page for details on internal buffering relating to '-u'
    -v : verbose (trace import statements); also PYTHONVERBOSE=x
    can be supplied multiple times to increase verbosity
    -V : print the Python version number and exit (also --version)
    -W arg : warning control; arg is action:message:category:module:lineno
    -x : skip first line of source, allowing use of non-Unix forms of #!cmd
    -3 : warn about Python 3.x incompatibilities
    file : program read from script file
    - : program read from stdin (default; interactive mode if a tty)
    arg ...: arguments passed to program in sys.argv[1:]

    Other environment variables:
    PYTHONSTARTUP: file executed on interactive startup (no default)
    PYTHONPATH : ';'-separated list of directories prefixed to the
    default module search path. The result is sys.path.
    PYTHONHOME : alternate <prefix> directory (or <prefix>;<exec_prefix>).
    The default module search path uses <prefix>\lib.
    PYTHONCASEOK : ignore case in 'import' statements (Windows).
    PYTHONIOENCODING: Encoding[:errors] used for stdin/stdout/stderr.


    HTH,

    Esmail
  • John Machin at May 24, 2009 at 1:58 pm

    On May 24, 11:01?pm, Stef Mientki wrote:
    hello,

    Moving my entire program section between windows and Ubuntu,
    sometimes causes problems, due to the existence of pyc-files
    What problems? Like avoiding having to recompile your .py files makes
    your app run too fast?
    (and probably because my program still has hard coded paths).

    Now I want get rid of the pyc-files,
    Which does nothing for the hard-coded paths issue ...
  • Pythoncurious at May 24, 2009 at 5:09 pm

    On May 24, 3:58?pm, John Machin wrote:
    What problems? Like avoiding having to recompile your .py files makes
    your app run too fast?
    There are real problems with this. I'm having similar problems when
    switching
    between Solaris and Windows.
    The code is in clearcase, which uses some sort of network file
    system.
    In our case, that means that I'll be accessing the same directories
    from both
    platforms, so .pyc-files from one platform will be visible when I run
    the
    code on the other platform.

    The .pyc-file will contain a string pointing to the file is was built
    from.
    The path to that file will be different under different platforms, so
    when the
    string is used, there will be error printouts.
    At least I think that's the problem, the error printouts contains
    paths that
    are only valid on the other platform.
    I don't have access to those computers atm, so I can't show the exact
    message.

    The only solution I've seen is to make sure to clean up the .pyc files
    each
    time I switch platform.
  • John Machin at May 24, 2009 at 10:07 pm

    On May 25, 3:09?am, pythoncuri... at gmail.com wrote:
    On May 24, 3:58?pm, John Machin wrote:


    What problems? Like avoiding having to recompile your .py files makes
    your app run too fast?
    There are real problems with this. I'm having similar problems when
    switching
    between Solaris and Windows.
    The code is in clearcase, which uses some sort of network file
    system.
    In our case, that means that I'll be accessing the same directories
    from both
    platforms, so .pyc-files from one platform will be visible when I run
    the
    code on the other platform.
    "switching" scarcely seems to be the right description. You appear to
    be running the same code from one repository simultaneously available
    to two different platforms.

    Try this: Instead of running your code straight from your repository,
    set up a run-time "launch pad" in an appropriate place for each
    platform. When there is a new release of your app, install it in each
    launchpad.


    The .pyc-file will contain a string pointing to the file is was built
    from.
    The path to that file will be different under different platforms, so
    when the
    string is used, there will be error printouts.
    At least I think that's the problem, the error printouts contains
    paths that
    are only valid on the other platform.
    I don't have access to those computers atm, so I can't show the exact
    message.

    The only solution I've seen is to make sure to clean up the .pyc files
    each
    time I switch platform.
    I'm a bit lost here ... your clearcase repository uses a mysterious
    "some sort of network file system" to which you have access
    permissions which allow you to delete individual files??
  • Pythoncurious at May 25, 2009 at 8:04 pm

    On May 25, 12:07?am, John Machin wrote:

    "switching" scarcely seems to be the right description. You appear to
    be running the same code from one repository simultaneously available
    to two different platforms.

    Try this: Instead of running your code straight from your repository,
    set up a run-time "launch pad" in an appropriate place for each
    platform. When there is a new release of your app, install it in each
    launchpad.
    For releases, that's a good enough idea. But while developing and
    wanting to run units tests, it's sort of a big overhead. Especially
    since I tend to run the unit test quite frequently.
    I'm a bit lost here ... your clearcase repository uses a mysterious
    "some sort of network file system" to which you have access
    permissions which allow you to delete individualfiles??
    Apparently I don't explain things well. :)
    Others have explained it better in this thread or you can have a look
    at http://en.wikipedia.org/wiki/Rational_ClearCase.
  • John Machin at May 25, 2009 at 11:52 pm

    On May 26, 6:04?am, pythoncuri... at gmail.com wrote:
    On May 25, 12:07?am, John Machin wrote:

    "switching" scarcely seems to be the right description. You appear to
    be running the same code from one repository simultaneously available
    to two different platforms.
    Try this: Instead of running your code straight from your repository,
    set up a run-time "launch pad" in an appropriate place for each
    platform. When there is a new release of your app, install it in each
    launchpad.
    For releases, that's a good enough idea. But while developing and
    wanting to run units tests, it's sort of a big overhead. Especially
    since I tend to run the unit test quite frequently.
    Oh .. so there's no concept of copying only the changed files??
  • Dave Angel at May 24, 2009 at 10:08 pm

    pythoncurious at gmail.com wrote:
    On May 24, 3:58 pm, John Machin wrote:

    What problems? Like avoiding having to recompile your .py files makes
    your app run too fast?
    There are real problems with this. I'm having similar problems when
    switching
    between Solaris and Windows.
    The code is in clearcase, which uses some sort of network file
    system.
    In our case, that means that I'll be accessing the same directories
    from both
    platforms, so .pyc-files from one platform will be visible when I run
    the
    code on the other platform.

    The .pyc-file will contain a string pointing to the file is was built
    from.
    The path to that file will be different under different platforms, so
    when the
    string is used, there will be error printouts.
    At least I think that's the problem, the error printouts contains
    paths that
    are only valid on the other platform.
    I don't have access to those computers atm, so I can't show the exact
    message.

    The only solution I've seen is to make sure to clean up the .pyc files
    each
    time I switch platform.
    Is Clearcase still around? I hope it works better than it did in 1992.

    Somebody else has already pointed out that you can tell Python not to
    create those files (during your development stages).

    But if that won't work for some reason, perhaps you can do something
    with symbolic links. I remember that RCS, for example, required that
    the archives be located in a directory immediately below the one with
    the sources. So in order to share those archives, you made the
    subdirectory actually a link to a common network location.

    Your case would seem to be the opposite. But I don't know enough about
    the current state of either Solaris or Clearcase to know the best answer.

    Perhaps Clearcase supports some form of "exclusion" parameter, wherein
    you say not to do version control on files with certain patterns, like .pyc
  • JKPeck at May 24, 2009 at 10:51 pm

    On May 24, 4:08?pm, Dave Angel wrote:
    pythoncuri... at gmail.com wrote:
    On May 24, 3:58 pm, John Machin wrote:

    What problems? Like avoiding having to recompile your .py files makes
    your app run too fast?
    There are real problems with this. I'm having similar problems when
    switching
    between Solaris and Windows.
    The code is in clearcase, which uses some sort of network file
    system.
    In our case, that means that I'll be accessing the same directories
    from both
    platforms, so .pyc-files from one platform will be visible when I run
    the
    code on the other platform.
    The .pyc-file will contain a string pointing to the file is was built
    from.
    The path to that file will be different under different platforms, so
    when the
    string is used, there will be error printouts.
    At least I think that's the problem, the error printouts contains
    paths that
    are only valid on the other platform.
    I don't have access to those computers atm, so I can't show the exact
    message.
    The only solution I've seen is to make sure to clean up the .pyc files
    each
    time I switch platform.
    Is Clearcase still around? ?I hope it works better than it did in 1992.

    Somebody else has already pointed out that you can tell Python not to
    create those files (during your development stages).

    But if that won't work for some reason, perhaps you can do something
    with symbolic links. ?I remember that RCS, for example, required that
    the archives be located in a directory immediately below the one with
    the sources. ?So in order to share those archives, you made the
    subdirectory actually a link to a common network location.

    Your case would seem to be the opposite. ?But I don't know enough about
    the current state of either Solaris or Clearcase to know the best answer.

    Perhaps Clearcase supports some form of "exclusion" parameter, wherein
    you say not to do version control on files with certain patterns, like .pyc
    ClearCase gives you tremendous control over what can be seen at any
    point. Assuming that you are using dynamic views, the simplest way to
    fix this is to use a different view for each platform. They would
    both be able to see the py files (although you could control that with
    the configspec)as checked in, but the pyc files would not be checked
    in and would automatically be view private. So with two different
    views, each platform would only see its own pyc files.

    HTH,
    Jon Peck
  • Pythoncurious at May 25, 2009 at 8:00 pm

    On May 25, 12:08?am, Dave Angel wrote:
    Is Clearcase still around? ?I hope it works better than it did in 1992.
    I don't know how it worked back then, but if it's worse than today, I
    don't know how they ever managed to get people to use it. I'm not a
    fan and I don't have a choice about using it :)
    Somebody else has already pointed out that you can tell Python not to
    create thosefiles(during your development stages).
    Yes, that's probably the best way to sort out the problems.
    It would of course be nice if python could detect that these files
    aren't usable and just regenerate them. I realise it might not be a
    trivial feature to implement though.
    Perhaps Clearcase supports some form of "exclusion" parameter, wherein
    you say not to do version control onfileswith certain patterns, like .pyc
    They're not actually version controlled, but "view private" as someone
    described. This means I'll see them whenever I'm using the view they
    were created in.
  • Gabriel Genellina at May 26, 2009 at 7:20 am

    En Mon, 25 May 2009 17:00:00 -0300, <pythoncurious at gmail.com> escribi?:
    On May 25, 12:08?am, Dave Angel wrote:

    Somebody else has already pointed out that you can tell Python not to
    create thosefiles(during your development stages).
    Yes, that's probably the best way to sort out the problems.
    It would of course be nice if python could detect that these files
    aren't usable and just regenerate them. I realise it might not be a
    trivial feature to implement though.
    .pyc files are platform independent; you *can* use the same .pyc files
    both on Windows and Linux.
    A .pyc/.pyo file contains two bits of info:
    - a magic number that depends on the Python version that created it
    - the last-modification-time of the corresponding .py source
    When you import a module, both numbers are read from the .pyc file. If the
    magic number matches the version of the running Python, and the
    modification time matches the one of the .py file (or no .py file exists),
    then the .pyc file is used. In any other case, the .py file is read,
    compiled, and a .pyc file is written -- if possible.

    So, if you ensure that both platforms "see" the same
    last-modification-time, and you use the same Python version on both
    platforms, then .pyc files may be shared. If not, each time you switch
    platforms all .pyc files will be re-created (option -B avoids that, as
    already noted).

    I any case, I don't think leaving the wrong .pyc files should create any
    problem (except some delay when the application starts, and unless you've
    found a bug, of course)

    --
    Gabriel Genellina
  • Gabriel Genellina at Jun 4, 2009 at 6:31 am
    [please keep the discussion on the list]
    may be python need a parameter to regenerate .pyo/pyc explicit ,not
    depending on magic number and modification time.
    but i just wander if you simply just clear all .pyc than generate in
    one system manually, can than program run without error in another
    system?
    Yes, you can always remove the .pyc files before starting, forcing a
    recompile. Also, see the compileall module.

    Provided that both systems use the same Python version, you can precompile
    .pyc files on one and copy them to the other. Even more, some people
    deploy only .pyc files.

    --
    Gabriel Genellina
  • David Lyon at May 24, 2009 at 11:21 pm

    On Sun, 24 May 2009 15:01:51 +0200, Stef Mientki wrote:
    hello,

    Moving my entire program section between windows and Ubuntu,
    sometimes causes problems, due to the existence of pyc-files
    (and probably because my program still has hard coded paths).

    Is there a way to prevent generating pyc-files ?
    Or is there a way to redirect the generated pyc-files to a dedicated
    location ?
    Yes.. I see your problem...

    try something else.... linking the files in linux...

    Try this:

    - create a windows "run-directory" as in 'mkdir win-run'

    - create symbolic link in the win-run directory for each script

    - 'cd win-run'

    - 'ln -s /home/user/mypythondir/hello.py hello.py'

    - for every source file...


    Now you'll be able to run both...

    Under windows the .pyc files will be created in the
    win-run directory and under linux in /home/user/mypythondir

    David
  • Terry Reedy at May 25, 2009 at 1:14 am

    Stef Mientki wrote:

    btw, What commandline switches are available for python ?
    The set of documents on the site and at least on Windows comes with
    Using Python. First chapter: Command Line ....
  • Steven D'Aprano at May 25, 2009 at 2:45 am

    On Sun, 24 May 2009 15:01:51 +0200, Stef Mientki wrote:

    Is there a way to prevent generating pyc-files ?
    Put the .py files in a read-only directory.

    Or is there a way to
    redirect the generated pyc-files to a dedicated location ?
    No.

    btw, What commandline switches are available for python ? (googling
    didn't give me any relevant hits )
    python --help


    at the shell (not Python) prompt.



    --
    Steven

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppython-list @
categoriespython
postedMay 24, '09 at 1:01p
activeJun 4, '09 at 6:31a
posts16
users11
websitepython.org

People

Translate

site design / logo © 2022 Grokbase