FAQ

On Jun 29, 12:20?pm, Javier Collado wrote:
Hello,

I would like to be able to run the main script in a python project
from both the source tree and the path in which it's installed on
Ubuntu. The script, among other things, imports a package which in
turns makes use of some data files that contains some metadata that is
needed in xml format.

The source tree has an structure such as this one:
setup.py
debian/ (packaging files)
src/ (source code)
src/lib (package files)
src/data (data files)
src/bin (main script)

However, when the project is installed using setup.py install, the
directory structure is approximately this way:
/usr/local/bin (main script)
/usr/local/share/<project_name> (data files)
/usr/local/lib/python2.x/dist-packages/<project_name> (library files)

And when installing the code through a package, the structure is the
same one, but removing "local".

Hence, the data files aren't always in the same relative directories
depending on we're executing code from the source tree or from the
installation. To make it possible to run the code from both places,
I've seen different approaches:
- distutils trick in setup.py to modify the installed script (i.e.
changing a global variable value) so that it has a reference to the
data files location.
- Heuristic in the package code to detect when it's being executed
from the source tree and when it has been the installed
- Just using an environment variable that the user must set according
to his needs

I guess that there are other options, for example, maybe using
buildout. What would you say it's the best/more elegant option to
solve this problem?

Best regards,
? ?Javier
It's kludgey, but one option may be to try and use __file__ to figure
out where the script is installed. Something like os.path.dirname
(os.path.abspath(__file__)) could tell you if it's in src/ or in the
bin/ directory, and then data files could be found in the appropriate
place.

I like the distutils/variable option better. Your script is more
likely to still behave correctly when copied to another directory.
Plus its code definitely remains cleaner.

Search Discussions

  • Carl Banks at Jul 1, 2009 at 6:04 am

    On Jun 29, 9:20?am, Javier Collado wrote:
    I've seen different approaches:
    - distutils trick in setup.py to modify the installed script (i.e.
    changing a global variable value) so that it has a reference to the
    data files location.

    One of my biggest complaints about distutils is that it doesn't do
    this, a limitation made worse by the fact that distutils allows you to
    specify an alternate data file directory, but your scripts have no way
    to know what that alternate directory is installed. Which really
    limits the usefulness of that functionality.

    The most common way I've seen people work around this issue is to
    throw their data files into the package directories. Yuck.

    At one point I hacked up a setup.py file to look in the distutils data
    structure and pull out the data install location, and wrote out a
    trivial python file listing that location. (The distutils build data
    structure is helpfully returned by setup function.) I never felt good
    about it, and it wouldn't work if the install was done in steps (such
    as build as user, install as superuser).

    If you care to navigate the murky internals of distutils to do this in
    a more elegant way, more power to you, and if so I'd recommend doing
    it that way.

    - Heuristic in the package code to detect when it's being executed
    from the source tree and when it has been the installed
    - Just using an environment variable that the user must set according
    to his needs
    I guess I'd combine these two. Make a sensible guess about where the
    data files are by checking out the environment, but if the data files
    aren't there (such as if the user installs to a different data
    location) then they are responsible for setting an env variable or
    config option.


    I guess that there are other options, for example, maybe using
    buildout. What would you say it's the best/more elegant option to
    solve this problem?
    Another option is not to use distutils at all. If you are writing an
    application, I think that would be a good idea. I don't think
    applications really need to be located in site-packages.

    If you have any C-extensions it might complicate this.


    Carl Banks
  • Robert Kern at Jul 1, 2009 at 4:17 pm

    On 2009-07-01 01:04, Carl Banks wrote:
    On Jun 29, 9:20 am, Javier Colladowrote:
    I've seen different approaches:
    - distutils trick in setup.py to modify the installed script (i.e.
    changing a global variable value) so that it has a reference to the
    data files location.

    One of my biggest complaints about distutils is that it doesn't do
    this, a limitation made worse by the fact that distutils allows you to
    specify an alternate data file directory, but your scripts have no way
    to know what that alternate directory is installed. Which really
    limits the usefulness of that functionality.

    The most common way I've seen people work around this issue is to
    throw their data files into the package directories. Yuck.
    Huh. I always found that to be a more elegant solution than hardcoding the data
    location into the program at install-time.

    --
    Robert Kern

    "I have come to believe that the whole world is an enigma, a harmless enigma
    that is made terrible by our own mad attempt to interpret it as though it had
    an underlying truth."
    -- Umberto Eco
  • Lawrence D'Oliveiro at Jul 2, 2009 at 5:53 am
    In message <mailman.2428.1246465065.8015.python-list at python.org>, Robert
    Kern wrote:
    On 2009-07-01 01:04, Carl Banks wrote:

    The most common way I've seen people work around this issue is to
    throw their data files into the package directories. Yuck.
    Huh. I always found that to be a more elegant solution than hardcoding the
    data location into the program at install-time.
    Think in terms of a portable OS, written to run on multiple architectures.
    Read-only data is architecture-dependent, that's why it pays to separate it
    from the code.

    Otherwise you end up with a system that needs a reinstall every time the
    hardware changes.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppython-list @
categoriespython
postedJul 1, '09 at 4:52a
activeJul 2, '09 at 5:53a
posts4
users4
websitepython.org

People

Translate

site design / logo © 2022 Grokbase