FAQ
With Python 3.5 shipping an embeddable copy of the interpreter on
Windows, I thought I'd try out a simple embedded interpreter as an
experiment. I wanted to use the limited API, as I'd rather it were
easy to upgrade the interpreter without recompiling the embedding app.


But the "Very high-level embedding" example in the docs doesn't
compile with the limited API.


#include <Python.h>


int
main(int argc, char *argv[])
{
     wchar_t *program = Py_DecodeLocale(argv[0], NULL);
     if (program == NULL) {
         fprintf(stderr, "Fatal error: cannot decode argv[0]\n");
         exit(1);
     }
     Py_SetProgramName(program); /* optional but recommended */
     Py_Initialize();
     PyRun_SimpleString("from time import time,ctime\n"
                        "print('Today is', ctime(time()))\n");
     Py_Finalize();
     PyMem_RawFree(program);
     return 0;
}


The Py_DecodeLocale/Py_SetProgramName/PyMem_RawFree bit can probably
be replaced by a Py_SetProgramName call specifying a static value,
it's not exactly crucial. (Py_DecodeLocale appears to be defined as in
the limited API by the headers, but not exported from python3.dll, by
the way, which implies that something's out of sync).


But PyRun_SimpleString doesn't appear to be exposed in the limited
API, even though https://docs.python.org/3/c-api/veryhigh.html doesn't
mention this, and https://docs.python.org/3/c-api/stable.html says
that functions not part of the stable API will be marked as such.


I dumped out the exported symbols from python3.dll, which is the
simplest way I could think of finding out what is in the limited API
(it's hardly user friendly, but never mind). And frustratingly, none
of the very high level PyRun_XXX APIs are available.


At this point, I think I'll probably just give up and use the full
API, but it does make me question whether the limited API is actually
usable as it stands.


I was hoping to be able to suggest as an application bundling option
that people could write a trivial wrapper script in C to fire up a
Python script, and bundle that along with its dependencies and the
embeddable Python distribution. Looks like that's doable, but only
using the full API, which makes upgrading the bundled Python
interpreter a bit messier. Ah, well, no huge loss :-(


But after this experiment, I do wonder - is the limited API really a
viable option for embedders?


Paul

Search Discussions

  • Nick Coghlan at May 28, 2015 at 2:28 pm

    On 29 May 2015 at 00:11, Paul Moore wrote:
    I was hoping to be able to suggest as an application bundling option
    that people could write a trivial wrapper script in C to fire up a
    Python script, and bundle that along with its dependencies and the
    embeddable Python distribution. Looks like that's doable, but only
    using the full API, which makes upgrading the bundled Python
    interpreter a bit messier. Ah, well, no huge loss :-(

    But after this experiment, I do wonder - is the limited API really a
    viable option for embedders?

    I personally suspect the requirement to preserve source compatibility
    with Python 2 has meant that the limited ABI hasn't been exercised
    very well to date.


    As far as the high level embedding API goes, I expect it's just an
    oversight that it's missing from the stable ABI. There are some that
    *can't* be exposed (the ones that rely on FILE pointers), but the
    others should be OK.


    Cheers,
    Nick.




    --
    Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia
  • Steve Dower at May 28, 2015 at 2:28 pm
    Zach has a patch to automate putting the right exports in python3.dll, which I'm strongly in favor of, but it was rejected because people may have added APIs that aren't meant to be stable.


    Right now, you can #include a number of prototypes that aren't actually available because there are two places to update and so one (in this case, the DLL) doesn't get updated.


    I think the current plan is to remove everything not currently in the DLL from the stable ABI and force people to add them back manually. This way we can enable the generator without committing to a large set of new APIs.


    I don't have the issue number handy, but it should be near the top of the recently modified list.


    Cheers,
    Steve


    Top-posted from my Windows Phone
    ________________________________
    From: Paul Moore<mailto:p.f.moore@gmail.com>
    Sent: ?5/?28/?2015 7:12
    To: Python Dev<mailto:python-dev@python.org>
    Subject: [Python-Dev] Usability of the limited API


    With Python 3.5 shipping an embeddable copy of the interpreter on
    Windows, I thought I'd try out a simple embedded interpreter as an
    experiment. I wanted to use the limited API, as I'd rather it were
    easy to upgrade the interpreter without recompiling the embedding app.


    But the "Very high-level embedding" example in the docs doesn't
    compile with the limited API.


    #include <Python.h>


    int
    main(int argc, char *argv[])
    {
         wchar_t *program = Py_DecodeLocale(argv[0], NULL);
         if (program == NULL) {
             fprintf(stderr, "Fatal error: cannot decode argv[0]\n");
             exit(1);
         }
         Py_SetProgramName(program); /* optional but recommended */
         Py_Initialize();
         PyRun_SimpleString("from time import time,ctime\n"
                            "print('Today is', ctime(time()))\n");
         Py_Finalize();
         PyMem_RawFree(program);
         return 0;
    }


    The Py_DecodeLocale/Py_SetProgramName/PyMem_RawFree bit can probably
    be replaced by a Py_SetProgramName call specifying a static value,
    it's not exactly crucial. (Py_DecodeLocale appears to be defined as in
    the limited API by the headers, but not exported from python3.dll, by
    the way, which implies that something's out of sync).


    But PyRun_SimpleString doesn't appear to be exposed in the limited
    API, even though https://docs.python.org/3/c-api/veryhigh.html doesn't
    mention this, and https://docs.python.org/3/c-api/stable.html says
    that functions not part of the stable API will be marked as such.


    I dumped out the exported symbols from python3.dll, which is the
    simplest way I could think of finding out what is in the limited API
    (it's hardly user friendly, but never mind). And frustratingly, none
    of the very high level PyRun_XXX APIs are available.


    At this point, I think I'll probably just give up and use the full
    API, but it does make me question whether the limited API is actually
    usable as it stands.


    I was hoping to be able to suggest as an application bundling option
    that people could write a trivial wrapper script in C to fire up a
    Python script, and bundle that along with its dependencies and the
    embeddable Python distribution. Looks like that's doable, but only
    using the full API, which makes upgrading the bundled Python
    interpreter a bit messier. Ah, well, no huge loss :-(


    But after this experiment, I do wonder - is the limited API really a
    viable option for embedders?


    Paul
    _______________________________________________
    Python-Dev mailing list
    Python-Dev at python.org
    https://mail.python.org/mailman/listinfo/python-dev
    Unsubscribe: https://mail.python.org/mailman/options/python-dev/steve.dower%40microsoft.com
    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: <http://mail.python.org/pipermail/python-dev/attachments/20150528/1c2c474a/attachment.html>
  • Paul Moore at May 28, 2015 at 2:49 pm

    On 28 May 2015 at 15:28, Steve Dower wrote:
    I don't have the issue number handy, but it should be near the top of the
    recently modified list.

    I recall seeing that issue. I'm fine with that - getting the two in
    sync is obviously worth doing (and clearly in hand). I'm personally
    not sure whether automating the exposure of symbols is the correct
    approach, as I'm not sure people typically even consider the stable
    API when adding functions. Is the default (what you get if somebody
    just blindly adds a symbol with no thought for the stable API) to
    expose it or not? If the default is that it's not exposed, then
    automation seems reasonable, otherwise I'm not so sure.


    The bigger issue for me is that it looks like the stable API doesn't
    include functions that allow you to just run a script/file.


    At a minimum, PyRun_SimpleString should be available. I'd also like to
    see a variant of PyRun_SimpleFile that let you pass a filename (either
    a .py file, or a zipfile, or a directory name - basically what you can
    pass to the Python interpreter directly). You can sort of do it via
    "import runpy; runpy.run_path(filename)", but you get into all sorts
    of fun with requoting filenames, etc.


    With the fact that we're distributing an embeddable interpreter for
    Windows, I'd like to be able to promote it as a bundling option,
    easier to use than things like py2exe/cx_Freeze.


    Paul
  • Steve Dower at May 28, 2015 at 3:02 pm

    Paul Moore wrote:
    On 28 May 2015 at 15:28, Steve Dower wrote:
    I don't have the issue number handy, but it should be near the top of
    the recently modified list.
    I recall seeing that issue. I'm fine with that - getting the two in sync is
    obviously worth doing (and clearly in hand). I'm personally not sure whether
    automating the exposure of symbols is the correct approach, as I'm not sure
    people typically even consider the stable API when adding functions. Is the
    default (what you get if somebody just blindly adds a symbol with no thought for
    the stable API) to expose it or not? If the default is that it's not exposed,
    then automation seems reasonable, otherwise I'm not so sure.

    Now I'm at my desk, the issue is http://bugs.python.org/issue23903


    I believe new symbols are considered stable by default, so perhaps we actually want a test that will generate a C file that imports everything "stable" and will break the buildbots if someone adds something new without explicitly adding it to the list of stable functions? That sounds like the only way to make the extra overhead of two lists work. I guess we could also invert all the logic in the header files so that symbols are unstable by default.

    The bigger issue for me is that it looks like the stable API doesn't include
    functions that allow you to just run a script/file.

    I think a combination of Py_CompileString and PyEval_EvalCode is what you want here, though I can't think of any good reason not to have a stable helper method (or even a macro?) for this.

    At a minimum, PyRun_SimpleString should be available. I'd also like to see a
    variant of PyRun_SimpleFile that let you pass a filename (either a .py file, or
    a zipfile, or a directory name - basically what you can pass to the Python
    interpreter directly). You can sort of do it via "import runpy;
    runpy.run_path(filename)", but you get into all sorts of fun with requoting
    filenames, etc.

    With the fact that we're distributing an embeddable interpreter for Windows, I'd
    like to be able to promote it as a bundling option, easier to use than things
    like py2exe/cx_Freeze.

    Agreed. This is the point of the zip file :)


    Cheers,
    Steve

    Paul
  • Nick Coghlan at May 28, 2015 at 10:13 pm

    On 29 May 2015 01:04, "Steve Dower" wrote:
    Paul Moore wrote:
    On 28 May 2015 at 15:28, Steve Dower wrote:
    I don't have the issue number handy, but it should be near the top of
    the recently modified list.
    I recall seeing that issue. I'm fine with that - getting the two in
    sync is
    obviously worth doing (and clearly in hand). I'm personally not sure
    whether
    automating the exposure of symbols is the correct approach, as I'm not
    sure
    people typically even consider the stable API when adding functions. Is
    the
    default (what you get if somebody just blindly adds a symbol with no
    thought for
    the stable API) to expose it or not? If the default is that it's not
    exposed,
    then automation seems reasonable, otherwise I'm not so sure.
    Now I'm at my desk, the issue is http://bugs.python.org/issue23903

    I believe new symbols are considered stable by default, so perhaps we
    actually want a test that will generate a C file that imports everything
    "stable" and will break the buildbots if someone adds something new without
    explicitly adding it to the list of stable functions?


    The stable CPython ABI is actually tracked on
    http://upstream-tracker.org/versions/python_stable_api.html


    Ideally we'd be running those checks automatically as part of our own QA
    with http://ispras.linuxbase.org/index.php/ABI_compliance_checker, similar
    to Antoine's regular refleak checks.


    Cheers,
    Nick.
    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: <http://mail.python.org/pipermail/python-dev/attachments/20150529/a2ad1ade/attachment.html>

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppython-dev @
categoriespython
postedMay 28, '15 at 2:11p
activeMay 28, '15 at 10:13p
posts6
users3
websitepython.org

People

Translate

site design / logo © 2017 Grokbase