FAQ
Hi,

I'm extending some old Visual Studio 6 code to add embedded python
scripting. It works fine most of the time but some python function calls do
not work as expected.

The C++ code is a multithreaded MFC application. I was assuming that it was
GIL issues but I have tried using the manual locking (PyEval_SaveThread &
PyEval_RestoreThread) and what seems to be the current method
(PyGILState_Ensure & PyGILState_Release)

Here's the error I'm getting:

Traceback (most recent call last):
File "...scripts\receipt_parser.py", line 296, in
get_giftcard_purchase_value
details = extract_transaction_details_section(test)
File "...scripts\receipt_parser.py", line 204, in
extract_transaction_details_section
for line in base_details:
TypeError: expected string or Unicode object, NoneType found

base_details is a list of strings (I can just define it like
'base_details=["1","2","3"...]' on the line previous) and the code runs fine
when run from standard interpreter. Many other function calls work fine from
the embedded app.

I create and then Py_DECREF the function parameters and the return value
after each function call. The module import is created at C++ object
constructor and then Py_DECREF'd in the desctuctor

Anyone else had issues of this kind? My next try will be to use
sub-interpreters per thread.

Thanks,
Paul.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20100202/e0b1eb27/attachment.htm>

Search Discussions

  • Gabriel Genellina at Feb 1, 2010 at 10:59 pm

    En Mon, 01 Feb 2010 18:21:56 -0300, Paul <gobladoome at gmail.com> escribi?:

    I'm extending some old Visual Studio 6 code to add embedded python
    scripting. It works fine most of the time but some python function calls
    do
    not work as expected.

    The C++ code is a multithreaded MFC application. I was assuming that it
    was
    GIL issues but I have tried using the manual locking (PyEval_SaveThread &
    PyEval_RestoreThread) and what seems to be the current method
    (PyGILState_Ensure & PyGILState_Release)

    Here's the error I'm getting:

    Traceback (most recent call last):
    File "...scripts\receipt_parser.py", line 296, in
    get_giftcard_purchase_value
    details = extract_transaction_details_section(test)
    File "...scripts\receipt_parser.py", line 204, in
    extract_transaction_details_section
    for line in base_details:
    TypeError: expected string or Unicode object, NoneType found

    base_details is a list of strings (I can just define it like
    'base_details=["1","2","3"...]' on the line previous) and the code runs
    fine
    when run from standard interpreter. Many other function calls work fine
    from
    the embedded app.
    I create and then Py_DECREF the function parameters and the return value
    after each function call. The module import is created at C++ object
    constructor and then Py_DECREF'd in the desctuctor
    Usually, errors in reference count handling prevent objects from being
    destroyed (a memory leak) or generate a GPF when accessing an
    now-inexistent object. In principle I'd look elsewhere.
    Anyone else had issues of this kind?
    Hard to tell without more info. base_details is built in C++ code? Is it
    actually a list, or a subclass defined by you?
    A common error is to forget to check *every* API function call for errors,
    so errors get unnoticed for a while but are reported on the next check,
    which may happen in an entirely unrelated function.
    My next try will be to use
    sub-interpreters per thread.
    I would not do that - I'd try to *simplify* the code to test, not make it
    more complicated.
    Does it work in a single-threaded application?

    --
    Gabriel Genellina
  • Paul at Feb 2, 2010 at 5:26 am
    Thanks Gabriel,

    I've managed to get it working and so far stable...

    What wasn't working reliably:

    mycppclass
    mycppclass::mycppclass()
    m_mypymodule = PyImport_Import(pModuleName)

    mycppclass::~ mycppclass()
    Py_XDECREF(m_mypymodule)

    mycppclass::callpy(funcname, args...)
    PyTuple_SetItem * args
    PyCallable_Check(func)
    PyObject_CallObject(func)

    Current working version:

    mycppclass
    mycppclass::mycppclass()
    {}

    mycppclass::~ mycppclass()
    {}

    mycppclass::callpy(funcname, args...)
    m_mypymodule = PyImport_Import(pModuleName)

    pyargs = PyTuple_SetItem * args
    PyCallable_Check(func)
    PyObject_CallObject(func,pyargs)

    Py_XDECREF(m_mypymodule)

    So now the module is being imported each function call (luckily I don't have
    to worry about performance)

    I assume this means that the internal representation of the imported module
    is being corrupted by something. I found another person with a similar issue
    here:
    http://mail.python.org/pipermail/python-dev/2004-March/043306.html - that is
    a long time ago but another multi-threaded app.

    I'm happy to use the working method but I'd like to understand what is going
    on a bit more. Can anyone shed any further light?

    Regards,
    Paul.
    On Tue, Feb 2, 2010 at 11:59 AM, Gabriel Genellina wrote:

    En Mon, 01 Feb 2010 18:21:56 -0300, Paul <gobladoome at gmail.com> escribi?:


    I'm extending some old Visual Studio 6 code to add embedded python
    scripting. It works fine most of the time but some python function calls
    do
    not work as expected.

    The C++ code is a multithreaded MFC application. I was assuming that it
    was
    GIL issues but I have tried using the manual locking (PyEval_SaveThread &
    PyEval_RestoreThread) and what seems to be the current method
    (PyGILState_Ensure & PyGILState_Release)

    Here's the error I'm getting:

    Traceback (most recent call last):
    File "...scripts\receipt_parser.py", line 296, in
    get_giftcard_purchase_value
    details = extract_transaction_details_section(test)
    File "...scripts\receipt_parser.py", line 204, in
    extract_transaction_details_section
    for line in base_details:
    TypeError: expected string or Unicode object, NoneType found

    base_details is a list of strings (I can just define it like
    'base_details=["1","2","3"...]' on the line previous) and the code runs
    fine
    when run from standard interpreter. Many other function calls work fine
    from
    the embedded app.
    I create and then Py_DECREF the function parameters and the return value
    after each function call. The module import is created at C++ object
    constructor and then Py_DECREF'd in the desctuctor
    Usually, errors in reference count handling prevent objects from being
    destroyed (a memory leak) or generate a GPF when accessing an now-inexistent
    object. In principle I'd look elsewhere.


    Anyone else had issues of this kind?
    Hard to tell without more info. base_details is built in C++ code? Is it
    actually a list, or a subclass defined by you?
    A common error is to forget to check *every* API function call for errors,
    so errors get unnoticed for a while but are reported on the next check,
    which may happen in an entirely unrelated function.


    My next try will be to use
    sub-interpreters per thread.
    I would not do that - I'd try to *simplify* the code to test, not make it
    more complicated.
    Does it work in a single-threaded application?

    --
    Gabriel Genellina


    --
    http://mail.python.org/mailman/listinfo/python-list
    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: <http://mail.python.org/pipermail/python-list/attachments/20100202/49b95a9e/attachment-0001.htm>
  • Gabriel Genellina at Feb 2, 2010 at 11:30 pm

    En Tue, 02 Feb 2010 02:26:57 -0300, Paul <gobladoome at gmail.com> escribi?:

    I've managed to get it working and so far stable...
    Glad to see you finally made it work!
    Current working version:
    [...]
    mycppclass::callpy(funcname, args...)
    m_mypymodule = PyImport_Import(pModuleName)

    pyargs = PyTuple_SetItem * args
    PyCallable_Check(func)
    PyObject_CallObject(func,pyargs)

    Py_XDECREF(m_mypymodule)

    So now the module is being imported each function call (luckily I don't
    have
    to worry about performance)
    Remember that after the module is successfully imported by the first time,
    a subsequent import returns early, as soon as it finds the module in
    sys.modules[] -- the performance penalty shouldn't be so big.
    I assume this means that the internal representation of the imported
    module
    is being corrupted by something. I found another person with a similar
    issue
    here:
    http://mail.python.org/pipermail/python-dev/2004-March/043306.html -
    that is
    a long time ago but another multi-threaded app.
    I'm happy to use the working method but I'd like to understand what is
    going
    on a bit more. Can anyone shed any further light?
    Sorry, I cannot help on this. Seems to happen only with an embedded
    interpreter and a multithreaded application, and I've never used Python in
    that scenario.

    If you could trim your code to a minimal example that shows the faulty
    behavior, that would be great, so others can test it too and eventually
    determine what's going on.

    --
    Gabriel Genellina

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppython-list @
categoriespython
postedFeb 1, '10 at 9:21p
activeFeb 2, '10 at 11:30p
posts4
users2
websitepython.org

2 users in discussion

Paul: 2 posts Gabriel Genellina: 2 posts

People

Translate

site design / logo © 2022 Grokbase