FAQ
Hi,

I've been Googling around on a moderately common Windows Python problem:
a mismatch between the symbols a python extension thinks are available
and the contents of the associated DLL. Python users running into this
problem are greeted with:

import <somemodule>
"ImportError: DLL load failed: The specified procedure could not be found."

That's it: no mention of which dll nor which symbol is causing the
problem. Both these pieces of information are helpful/important in
resolving this class of problem: how does one get Python to dump this
debugging data?

(>python -v -d
doesn't help, BTW)

Regards,
Chris

Search Discussions

  • John Machin at Feb 23, 2009 at 6:38 am

    On Feb 23, 2:56?pm, Chris Cormie wrote:
    Hi,

    I've been Googling around on a moderately common Windows Python problem:
    a mismatch between the symbols a python extension thinks are available
    and the contents of the associated DLL. Python users running into this
    problem are greeted with:

    import <somemodule>
    "ImportError: DLL load failed: The specified procedure could not be found."

    That's it: no mention of which dll nor which symbol is causing the
    problem. Both these pieces of information are helpful/important in
    resolving this class of problem: how does one get Python to dump this
    debugging data?

    (>python -v -d
    doesn't help, BTW)
    Use -vv ... you need to know exactly which of possibly many
    incarnations of somemodule.pyd Python is trying to import from.

    Then get a copy of the Dependency Walker from http://www.dependencywalker.com/

    Open c:\the\failing\somemodule.pyd with the DW. Look at grumble
    messages in the bottom pane. Note: you often get 1 or 2 warning
    messages with a pyd that's not having problems. Scroll through all the
    modules in the middle pane, looking for grumbles.

    If that not-very-technical description [all I've ever needed] doesn't
    help, you'll need to read the DW help file (HTFF1K) or wait till
    someone who knows what they are doing comes along :-)

    By the way, I presume that you are aware that extensions on Windows
    are tied to a particular version of Python ... you would have got a
    different error message if you had a version mismatch (I think). If
    your problem is not obvious, come back here with:
    -- what version of Python you are running
    -- what DLLs you find in the middle pane whose names match
    (1) MSVC*.DLL
    (2) PYTHON??.DLL

    You may need to answer questions about the extension (e.g. compiler/
    linker options used) -- is it your extension or a 3rd party's?

    HTH,
    John
  • Chris Cormie at Feb 23, 2009 at 12:41 pm

    If that not-very-technical description [all I've ever needed] doesn't
    help, you'll need to read the DW help file (HTFF1K) or wait till
    someone who knows what they are doing comes along :-)
    LOL, I am that person :p
    Your technique works well and it does provide the information and it is
    a (roundabout) solution to this class of problems: thank you very much.
    It doesn't answer the original question so I'll restate it:

    How do you get *Python* to tell you the dll and the problem symbol? Not
    external tools, Python. Python at a low level is calling LoadLibrary and
    GetProcAddress to resolve symbols and the call fails. At that point it
    has the name of the dll and the problem symbol to hand and yet strangely
    only gives an opaque error message. How does one get Python to print out
    the faulty DLL and symbol?

    (Even if it means a debug build of Python and adding the printf
    yourself, I just want to know where we stand on this issue! :) )

    Best Regards,
    Chris
  • John Machin at Feb 23, 2009 at 1:05 pm

    On Feb 23, 11:41?pm, Chris Cormie wrote:
    If that not-very-technical description [all I've ever needed] doesn't
    help, you'll need to read the DW help file (HTFF1K) or wait till
    someone who knows what they are doing comes along :-)
    LOL, I am that person :p
    It wasn't apparent, and still isn't.
    Your technique works well and it does provide the information and it is
    a (roundabout) solution to this class of problems: ?thank you very much.
    It doesn't answer the original question so I'll restate it:

    How do you get *Python* to tell you the dll and the problem symbol? Not
    external tools, Python. Python at a low level is calling LoadLibrary and
    GetProcAddress to resolve symbols and the call fails. At that point it
    has the name of the dll and the problem symbol to hand and yet strangely
    only gives an opaque error message. How does one get Python to print out
    the faulty DLL and symbol?
    If you know all that, then the answer is to submit a patch, isn't it??

    (Even if it means a debug build of Python and adding the printf
    yourself,
    printf? Maybe you shouldn't submit a patch after all.
  • Chris Cormie at Feb 24, 2009 at 3:25 am

    John Machin wrote:
    On Feb 23, 11:41 pm, Chris Cormie wrote:
    If that not-very-technical description [all I've ever needed] doesn't
    help, you'll need to read the DW help file (HTFF1K) or wait till
    someone who knows what they are doing comes along :-)
    LOL, I am that person :p
    It wasn't apparent, and still isn't.
    Yikes, enough with the rudeness people. I'm asking a reasonable question
    in a good faith.
  • Gabriel Genellina at Feb 23, 2009 at 1:49 pm
    En Mon, 23 Feb 2009 10:41:20 -0200, Chris Cormie
    <ccormie at aussiemail.com.au> escribi?:
    If that not-very-technical description [all I've ever needed] doesn't
    help, you'll need to read the DW help file (HTFF1K) or wait till
    someone who knows what they are doing comes along :-)
    LOL, I am that person :p
    Your technique works well and it does provide the information and it is
    a (roundabout) solution to this class of problems: thank you very much.
    It doesn't answer the original question so I'll restate it:

    How do you get *Python* to tell you the dll and the problem symbol? Not
    external tools, Python. Python at a low level is calling LoadLibrary and
    GetProcAddress to resolve symbols and the call fails. At that point it
    has the name of the dll and the problem symbol to hand and yet strangely
    only gives an opaque error message. How does one get Python to print out
    the faulty DLL and symbol?
    You can't, because it isn't Python who's trying to load the symbol - the
    *only* symbol that Python attempts to import itself is "initFOO" from
    FOO.pyd, and when that fails you get an explicit message.
    The error you see must come from the extension itself, and propagates into
    Python as an ImportError.

    --
    Gabriel Genellina
  • Mark Hammond at Feb 23, 2009 at 9:49 pm

    On 23/02/2009 11:41 PM, Chris Cormie wrote:
    If that not-very-technical description [all I've ever needed] doesn't
    help, you'll need to read the DW help file (HTFF1K) or wait till
    someone who knows what they are doing comes along :-)
    LOL, I am that person :p
    LOL sounds right!
    How do you get *Python* to tell you the dll and the problem symbol? Not
    external tools, Python. Python at a low level is calling LoadLibrary and
    GetProcAddress to resolve symbols and the call fails.
    It is the LoadLibrary that is failing; it should be obvious that if it
    was a simple GetProcAddress that was failing, Python would simply throw
    an exception rather than displaying the ugly dialog box you see.

    The problem is that some *other* DLL in the chain of dependencies is
    failing to load; please re-adjust your perceptions of your own knowledge
    and re-read John's initial response - if you follow his instructions
    that should all become quite obvious.

    Mark
  • Chris Cormie at Feb 24, 2009 at 2:49 am

    Mark Hammond wrote:
    On 23/02/2009 11:41 PM, Chris Cormie wrote:

    If that not-very-technical description [all I've ever needed] doesn't
    help, you'll need to read the DW help file (HTFF1K) or wait till
    someone who knows what they are doing comes along :-)
    LOL, I am that person :p
    LOL sounds right!
    How do you get *Python* to tell you the dll and the problem symbol? Not
    external tools, Python. Python at a low level is calling LoadLibrary and
    GetProcAddress to resolve symbols and the call fails.
    It is the LoadLibrary that is failing; it should be obvious that if it
    was a simple GetProcAddress that was failing, Python would simply throw
    an exception rather than displaying the ugly dialog box you see.
    I'm talking about the whole class of errors when loading an extension,
    the LoadLibary succeeds but the GetProcAddress fails. Not one specific
    error, a class of errors and *by definition* from the original question
    LoadLibrary succeeds. That's the point: at some point in this common
    scenario we have the path to the dll and a symbol we are resolving, and
    the symbol name, yet for whatever reason the GetProcAddress lookup
    fails, resulting in a surprisingly uninformative errors message.

    Regards,
    Chris.
  • Chris Cormie at Feb 24, 2009 at 3:23 am

    Gabriel Genellina wrote:
    En Mon, 23 Feb 2009 10:41:20 -0200, Chris Cormie
    <ccormie at aussiemail.com.au> escribi?:
    If that not-very-technical description [all I've ever needed] doesn't
    help, you'll need to read the DW help file (HTFF1K) or wait till
    someone who knows what they are doing comes along :-)
    LOL, I am that person :p
    Your technique works well and it does provide the information and it
    is a (roundabout) solution to this class of problems: thank you very
    much. It doesn't answer the original question so I'll restate it:

    How do you get *Python* to tell you the dll and the problem symbol?
    Not external tools, Python. Python at a low level is calling
    LoadLibrary and GetProcAddress to resolve symbols and the call fails.
    At that point it has the name of the dll and the problem symbol to
    hand and yet strangely only gives an opaque error message. How does
    one get Python to print out the faulty DLL and symbol?
    You can't, because it isn't Python who's trying to load the symbol - the
    *only* symbol that Python attempts to import itself is "initFOO" from
    FOO.pyd, and when that fails you get an explicit message.
    The error you see must come from the extension itself, and propagates
    into Python as an ImportError.
    Right -- now we are getting somewhere! That makes a lot of sense.

    What I was expecting to see next was an explicit GetProcAddress() call
    in the extension code that was failing in initFOO() or elsewhere in the
    extension code. But there are no such calls.

    I think what might have been making this confusing is that the root of
    my example problem is link time.

    Let me explain by giving the specific example case I have to hand.
    I orginally got on to this class of opaque runtime errors caused by
    missing symbols through building the pycURL extension on MinGW. As of
    Python 2.6, Python links against the new msvcr90.dll C Runtime. MinGW
    maintains a set of .a files for system DLLs including the C Runtime.
    When building pycURL naturally I want to link against libmsvcr90.a so
    Python 2.6 and pycURL are using the same C Rutime. Problem was, MinGW's
    .a was wrong: it exports symbols that msvcr90.dll does not in fact
    possess. Building pycURL with the faulty libmsvcr90.a causes the opaque
    error message: "ImportError: DLL load failed"

    Now I expected this meant an explict LoadLibrary() GetProcAddress()
    sequence was failing in the extension but perhaps that's not what
    happens at all?
    Chris
  • Chris Cormie at Feb 24, 2009 at 3:54 am

    Mark Hammond wrote:
    On 23/02/2009 11:41 PM, Chris Cormie wrote:

    If that not-very-technical description [all I've ever needed] doesn't
    help, you'll need to read the DW help file (HTFF1K) or wait till
    someone who knows what they are doing comes along :-)
    LOL, I am that person :p
    LOL sounds right!
    How do you get *Python* to tell you the dll and the problem symbol? Not
    external tools, Python. Python at a low level is calling LoadLibrary and
    GetProcAddress to resolve symbols and the call fails.
    It is the LoadLibrary that is failing; it should be obvious that if it
    was a simple GetProcAddress that was failing, Python would simply throw
    an exception rather than displaying the ugly dialog box you see.

    The problem is that some *other* DLL in the chain of dependencies is
    failing to load; please re-adjust your perceptions of your own knowledge
    and re-read John's initial response - if you follow his instructions
    that should all become quite obvious.

    Mark
    I think an example would help rather than me jabbering on about
    LoadLibary/GetProcAddress .

    It really is a case of Python loads module x depending on DLL y that is
    missing symbol z, yet the error message is just "load x failed" when in
    theory it could be "load x failed: y doesn't have z".

    However I think I am confusing people (and myself!) by assuming it's
    always as explicit as a LoadLibary/GetProcAddress sequence that is
    failing when people see this error.

    Let me explain by giving the specific example case I have to hand.
    I originally got on to this class of opaque runtime errors caused by
    missing symbols through building the pycURL extension on MinGW. As of
    Python 2.6, Python links against the new msvcr90.dll C Runtime. MinGW
    maintains a set of .a files for system DLLs including the C Runtime.
    When building pycURL naturally I want to link against libmsvcr90.a so
    Python 2.6 and pycURL are using the same C Runtime. Problem was, MinGW's
    .a was wrong: it exports symbols that msvcr90.dll does not in fact
    possess. Building pycURL with the faulty libmsvcr90.a causes the opaque
    error message: "ImportError: DLL load failed"

    I think Gabriel's got the part of the answer: she said "it isn't Python
    who's trying to load the symbol - the *only* symbol that Python attempts
    to import itself is "initFOO" from FOO.pyd"

    So my next question is: given the above missing-symbol problem, is the
    system able/willing to report the bad symbol reference in the .pyd when
    Python attempts to import initFOO?

    Regards,
    Chris.
  • John Machin at Feb 24, 2009 at 5:01 am

    On Feb 24, 2:54?pm, Chris Cormie wrote:
    Mark Hammond wrote:
    On 23/02/2009 11:41 PM, Chris Cormie wrote:

    If that not-very-technical description [all I've ever needed] doesn't
    help, you'll need to read the DW help file (HTFF1K) or wait till
    someone who knows what they are doing comes along :-)
    LOL, I am that person :p
    LOL sounds right!
    How do you get *Python* to tell you the dll and the problem symbol? Not
    external tools, Python. Python at a low level is calling LoadLibrary and
    GetProcAddress to resolve symbols and the call fails.
    It is the LoadLibrary that is failing; it should be obvious that if it
    was a simple GetProcAddress that was failing, Python would simply throw
    an exception rather than displaying the ugly dialog box you see.
    The problem is that some *other* DLL in the chain of dependencies is
    failing to load; please re-adjust your perceptions of your own knowledge
    and re-read John's initial response - if you follow his instructions
    that should all become quite obvious.
    Mark
    I think an example would help rather than me jabbering on about
    LoadLibary/GetProcAddress .

    It really is a case of Python loads module x depending on DLL y that is
    missing symbol z, yet the error message is just "load x failed" when in
    theory it could be "load x failed: y doesn't have z".

    However I think I am confusing people (and myself!) by assuming it's
    Don't assume; read the [open] source code; see below.
    always as explicit as a LoadLibary/GetProcAddress sequence that is
    failing when people see this error.

    Let me explain by giving the specific example case I have to hand.
    I originally got on to this class of opaque runtime errors caused by
    missing symbols through building the pycURL extension on MinGW. As of
    Python 2.6, Python links against the new msvcr90.dll C Runtime. MinGW
    maintains a set of .a files for system DLLs including the C Runtime.
    When building pycURL naturally I want to link against libmsvcr90.a so
    Python 2.6 and pycURL are using the same C Runtime. Problem was, MinGW's
    ? .a was wrong: it exports symbols that msvcr90.dll does not in fact
    possess. Building pycURL with the faulty libmsvcr90.a causes the opaque
    error message: "ImportError: DLL load failed"

    I think Gabriel's got the part of the answer: she said ?"it isn't Python
    who's trying to load the symbol - the *only* symbol that Python attempts
    to import itself is "initFOO" from FOO.pyd"

    So my next question is: given the above missing-symbol problem, is the
    system able/willing to report the bad symbol reference in the .pyd when
    Python attempts to import initFOO?
    My reading of the Python 2.6.1 version of dynload_win.c tells me that:

    1. It uses LoadLibraryEx() to attempt to load the pyd given its full
    pathname

    2. If that fails, it gets the Windows error code using GetLastError()

    3. It gets the Windows message text using FormatMessage()

    4. It assembles the message as "DLL load failed: " plus the Windows
    message text (less any trailing "\r\n").

    Note that if the pyd load succeeds, it rummages in memory to find the
    pythonxx.dll used by the pyd so that it can be checked for
    consistency. Then and only then it uses GetProcAddress() to get the
    address of the initFOO function.

    Do you know a way of getting more info out of Windows than GetLastError
    ()?
  • Chris Cormie at Feb 24, 2009 at 8:00 am

    My reading of the Python 2.6.1 version of dynload_win.c tells me that:

    1. It uses LoadLibraryEx() to attempt to load the pyd given its full
    pathname

    2. If that fails, it gets the Windows error code using GetLastError()

    3. It gets the Windows message text using FormatMessage()

    4. It assembles the message as "DLL load failed: " plus the Windows
    message text (less any trailing "\r\n").

    Note that if the pyd load succeeds, it rummages in memory to find the
    pythonxx.dll used by the pyd so that it can be checked for
    consistency. Then and only then it uses GetProcAddress() to get the
    address of the initFOO function.

    Do you know a way of getting more info out of Windows than GetLastError
    ()?
    Thank you for your help, it is much appreciated:

    In summary:

    Q) How can one get Python to tell you which symbol is causing a problem
    when it loads an extension with a bad reference to a symbol in a DLL it
    uses?

    A) There is no such way: you *must* use external tools: When a symbol is
    missing form a DLL that a Python extension depends on, the failure will
    be picked up in Python immediately on attempting to load the extension:
    the extension code is never reached and there is no specific attempt
    made in Python to resolve the problem symbol. further, The normal
    platform APIs lack the ability to determine the cause of the load failure.
    Do you know a way of getting more info out of Windows than GetLastError
    ()?
    No, but presumably the ability to determine in code that a DLL has a bad
    reference and which symbol is bad exists because the depends program is
    able to do it. Of course it's a separate question as to whether Python
    wants to load itself down with complex platform specific code for a
    small corner case.

    Best Regards,
    Chris.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppython-list @
categoriespython
postedFeb 23, '09 at 3:56a
activeFeb 24, '09 at 8:00a
posts12
users4
websitepython.org

People

Translate

site design / logo © 2021 Grokbase