FAQ
Howdy all,

Python's standard library has modules for configuration file parsing
(configparser) and command-line argument parsing (optparse, argparse). I
want to write a program that does both, but also:

* Has a cascade of options: default option values, overridden by config
file options, overridden by command-line options.

* Reads a different, or even additional, configuration file if specified
on the command-line (e.g. --config-file foo.conf) and yet still obeys
the above cascade.

* Allows a single definition of an option (e.g. logging level) to define
the same option for parsing from configuration files and the command
line.

* Unifies the parsed options into a single collection for the rest of
the program to access without caring where they came from.

How can I achieve this with minimum deviation from the Python standard
library?


(For anyone interested in gaining StackOverflow points, I'm also asking
this as a question there so feel free to post answers on that site
<URL:http://stackoverflow.com/questions/6133517/parse-config-file-and-command-line-arguments-to-get-a-single-collection-of-optio>.)

--
\ ?Apologize, v. To lay the foundation for a future offense.? |
`\ ?Ambrose Bierce, _The Devil's Dictionary_, 1906 |
_o__) |
Ben Finney

Search Discussions

  • Raymond Hettinger at May 26, 2011 at 5:15 am

    On May 25, 9:38?pm, Ben Finney wrote:
    Howdy all,

    Python's standard library has modules for configuration file parsing
    (configparser) and command-line argument parsing (optparse, argparse). I
    want to write a program that does both, but also:

    * Has a cascade of options: default option values, overridden by config
    ? file options, overridden by command-line options.

    * Reads a different, or even additional, configuration file if specified
    ? on the command-line (e.g. --config-file foo.conf) and yet still obeys
    ? the above cascade.

    * Allows a single definition of an option (e.g. logging level) to define
    ? the same option for parsing from configuration files and the command
    ? line.

    * Unifies the parsed options into a single collection for the rest of
    ? the program to access without caring where they came from.

    How can I achieve this with minimum deviation from the Python standard
    library?
    One thought is start with something like ChainMap,
    http://code.activestate.com/recipes/305268-chained-map-lookups/?in=user-178123
    , or some variant to unify multiple mapping objects into a single
    prioritized collection. A mapping for command line args can be made
    by using vars() on an argparse namespace to create a dictionary.
    ConfigParser's mapping is accessible via its get() method. With a
    ChainMap style object you can add other option sources such as
    os.environ. This should get you started on your grand unified, do-
    everything-at-once vision with minimal deviation from the standard
    library.

    Raymond
  • Rzed at May 28, 2011 at 11:12 pm
    Ben Finney <ben at benfinney.id.au> wrote in
    news:87k4deaxfc.fsf at benfinney.id.au:
    Howdy all,

    Python's standard library has modules for configuration file
    parsing (configparser) and command-line argument parsing
    (optparse, argparse). I want to write a program that does both,
    but also:

    * Has a cascade of options: default option values, overridden by
    config
    file options, overridden by command-line options.

    * Reads a different, or even additional, configuration file if
    specified
    on the command-line (e.g. --config-file foo.conf) and yet still
    obeys the above cascade.

    * Allows a single definition of an option (e.g. logging level) to
    define
    the same option for parsing from configuration files and the
    command line.

    * Unifies the parsed options into a single collection for the
    rest of
    the program to access without caring where they came from.

    How can I achieve this with minimum deviation from the Python
    standard library?


    (For anyone interested in gaining StackOverflow points, I'm also
    asking this as a question there so feel free to post answers on
    that site
    <URL:http://stackoverflow.com/questions/6133517/parse-config-file-
    and-command-line-arguments-to-get-a-single-collection-of-optio>.)
    This seems vaguely similar to a module I wrote and use all the time.
    It allows default value specification, categorization of command-line
    options, in-line parsing of further spec file[s] and overrides of
    values in the the sequence you define, pretty much. It doesn't deal
    with the "logging level" item. I'm not sure what that would mean. The
    idea of the module is to create a namespace object (though it could
    as easily be a dict) that contains whatever values are specified.

    in the program it would be invoked like this:
    ctx = Cmdline( outfname='test.out', size=(250,400), ... ) (or
    whatever).

    The command line can contain switches, prefixed by hyphens, spec file
    names, prefixed by @, untagged names, or key=value pairs. The values
    will be parsed as (tuples), [lists], or {dicts}, ints, floats, or
    strings. I did not, I am ashamed to say, resist the temptation to
    guess. 'Ctx' will contain the result of all this. Switches, if any
    are in a list named ctx._switches, unadorned arguments are in a list
    named ctx._vars, and the other stuff is pretty much as you would
    expect. It expects configuration files to be in a sort of ini-file
    format.

    Here's an example:

    ---------------------------
    test.spec:
    log=test.log
    count
    size=(400,200)
    group1={key=value,a=alpha}
    group2={b¾ta,name=doogie howser}
    #
    temp=[1,2,5,11,22]
    sub=(al,becky,carl,diane,edwin,fran)

    ---------------------------
    test.py:
    from Cmdline import Cmdline
    c = Cmdline( count=5, log='pink.tank')
    c.show()

    ---------------------------
    python test.py log=friend.txt @test.spec count2 name=waldo
    ... yields:
    <type 'int'> count = 32
    <type 'tuple'> sub = ('al', 'becky', 'carl', 'diane', 'edwin',
    'fran')
    <type 'str'> log = 'test.log'
    <type 'list'> temp = [1, 2, 5, 11, 22]
    <type 'dict'> group1 = {'a': 'alpha', 'key': 'value'}
    <type 'dict'> group2 = {'b': 'beta', 'name': 'doogie howser'}
    <type 'tuple'> size = (400, 200)
    <type 'str'> name = 'waldo'

    ... while
    python @test.spec count2 name=waldo temp=Vitalis sub='' dream -k
    <type 'int'> count = 32
    <type 'list'> _vars = ['dream']
    <type 'str'> log = 'test.log'
    <type 'str'> temp = 'Vitalis'
    <type 'list'> _switches = ['k']
    <type 'str'> name = 'waldo'
    <type 'dict'> group1 = {'a': 'alpha', 'key': 'value'}
    <type 'dict'> group2 = {'b': 'beta', 'name': 'doogie howser'}
    <type 'tuple'> size = (400, 200)
    <type 'str'> sub = ''

    What the program does with the results is up to it, of course.

    --
    rzed

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppython-list @
categoriespython
postedMay 26, '11 at 4:38a
activeMay 28, '11 at 11:12p
posts3
users3
websitepython.org

People

Translate

site design / logo © 2022 Grokbase