FAQ
There is a minor bug in inspect.findsource
The starting line number of an examined class is searched by using
regular expressions. The problem of using regexps is that they can't
handle multiline strings correctly, whereas tokenizer can.

Here's an example where original inspect.findsource returns an incorrect
answer, whereas modified inspect2.findsource, that uses tokenizer
returns a right answer.

Python 2.2 (#28, Dec 21 2001, 12:21:22) [MSC 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
import inspect, inspect2, inspect_test
inspect.findsource(inspect_test.DummyClass)
(["'''\n", 'You should remember, that\n', 'class DummyClass can only
be\n',
'used for dummy purposes.\n', "'''\n", '\n', 'class DummyClass:\n',
' pass\n'], 2 )
inspect2.findsource(inspect_test.DummyClass)
(["'''\n", 'You should remember, that\n', 'class DummyClass can only
be\n',
'used for dummy purposes.\n', "'''\n", '\n', 'class DummyClass:\n',
' pass\n'], 7 )

The biggest drawback of using the tokenizer is, that is it very slow,
but it is
not a good thing either, that a library function returns an incorrect
answer.
Perhaps tokenizer should be implemented in C to achieve reasonable
speed.

## inspect_test.py begin ##
'''
You should remember, that
class DummyClass can only be
used for dummy purposes.
'''

class DummyClass:
pass
## inspect_test.py end ##



## original inspect.findsource begin ###
def findsource(object):
"""Return the entire source file and starting line number for an
object.

The argument may be a module, class, method, function, traceback,
frame,
or code object. The source code is returned as a list of all the
lines
in the file and the line number indexes a line in that list. An
IOError
is raised if the source code cannot be retrieved."""
try:
file = open(getsourcefile(object))
except (TypeError, IOError):
raise IOError, 'could not get source code'
lines = file.readlines()
file.close()

if ismodule(object):
return lines, 0

if isclass(object):
name = object.__name__
pat = re.compile(r'^\s*class\s*' + name + r'\b')
for i in range(len(lines)):
if pat.match(lines[i]): return lines, i
else: raise IOError, 'could not find class definition'

if ismethod(object):
object = object.im_func
if isfunction(object):
object = object.func_code
if istraceback(object):
object = object.tb_frame
if isframe(object):
object = object.f_code
if iscode(object):
if not hasattr(object, 'co_firstlineno'):
raise IOError, 'could not find function definition'
lnum = object.co_firstlineno - 1
pat = re.compile(r'^\s*def\s')
while lnum > 0:
if pat.match(lines[lnum]): break
lnum = lnum - 1
return lines, lnum
## original inspect.findsource end ###



## modified inspect2.findsource begin ##
def findsource(object):
"""Return the entire source file and starting line number for an
object.

The argument may be a module, class, method, function, traceback,
frame,
or code object. The source code is returned as a list of all the
lines
in the file and the line number indexes a line in that list. An
IOError
is raised if the source code cannot be retrieved."""
try:
file = open(getsourcefile(object))
except (TypeError, IOError):
raise IOError, 'could not get source code'
lines = file.readlines()
file.close()

if ismodule(object):
return lines, 0

if isclass(object):
name = object.__name__
# XXX that StringIO trick is really ugly
tokens =
tokenize.generate_tokens(StringIO.StringIO("".join(lines)).readline)
while 1:
try:
if tokens.next()[1] == "class":
nexttoken = tokens.next()
if nexttoken[1] == name:
return lines, nexttoken[2][0]
except StopIteration:
raise IOError, 'could not find class definition'

if ismethod(object):
object = object.im_func
if isfunction(object):
object = object.func_code
if istraceback(object):
object = object.tb_frame
if isframe(object):
object = object.f_code
if iscode(object):
if not hasattr(object, 'co_firstlineno'):
raise IOError, 'could not find function definition'
lnum = object.co_firstlineno - 1
pat = re.compile(r'^\s*def\s')
while lnum > 0:
if pat.match(lines[lnum]): break
lnum = lnum - 1
return lines, lnum
## modified inspect2.findsource end ##

Search Discussions

  • Fernando Pérez at Jan 10, 2002 at 12:15 am
    [snipped]

    Why don't you submit it as a patch? Just go to sourceforge and put it in.
    That's how python gets better...

    Cheers,

    f.
  • Joonas Paalasmaa at Jan 11, 2002 at 8:39 pm

    Fernando P?rez wrote:
    [snipped]

    Why don't you submit it as a patch? Just go to sourceforge and put it in.
    That's how python gets better...
    Because I think that the tokenizer solution is too slow. Below are shown
    some test
    results that are claiming, that the tokenizer version runs over 7 times
    slower than the
    original. What do you think? Should I suggest a patch?

    Python 2.2 (#28, Dec 21 2001, 12:21:22) [MSC 32 bit (Intel)] on win32
    Type "help", "copyright", "credits" or "license" for more information.
    import profile, inspect, inspect2, cgi
    profile.run("inspect.findsource(cgi.FormContent)")
    138 function calls (134 primitive calls) in 0.327 CPU seconds

    Ordered by: standard name

    ncalls tottime percall cumtime percall filename:lineno(function)
    1 0.001 0.001 0.200 0.200 <string>:1(?)
    1 0.000 0.000 0.000 0.000 inspect.py:279(getfile)
    1 0.001 0.001 0.002 0.002
    inspect.py:318(getsourcefile)
    2 0.000 0.000 0.000 0.000 inspect.py:33(ismodule)
    1 0.159 0.159 0.198 0.198 inspect.py:368(findsource)
    2 0.000 0.000 0.000 0.000 inspect.py:41(isclass)
    1 0.001 0.001 0.001 0.001 ntpath.py:244(exists)
    1 0.127 0.127 0.327 0.327
    profile:0(inspect.findsource(cgi
    .FormContent))
    0 0.000 0.000 profile:0(profiler)
    1 0.000 0.000 0.037 0.037 sre.py:176(compile)
    1 0.000 0.000 0.036 0.036 sre.py:215(_compile)
    2 0.000 0.000 0.001 0.000
    sre_compile.py:145(_compile_charse
    t)
    2 0.000 0.000 0.000 0.000
    sre_compile.py:174(_optimize_chars
    et)
    3/1 0.018 0.006 0.020 0.020
    sre_compile.py:21(_compile)
    2 0.000 0.000 0.001 0.000
    sre_compile.py:293(_simple)
    1 0.000 0.000 0.002 0.002
    sre_compile.py:300(_compile_info)
    1 0.000 0.000 0.022 0.022 sre_compile.py:409(_code)
    1 0.000 0.000 0.036 0.036
    sre_compile.py:424(compile)
    8 0.000 0.000 0.000 0.000 sre_parse.py:133(__len__)
    10 0.000 0.000 0.000 0.000
    sre_parse.py:137(__getitem__)
    2 0.000 0.000 0.000 0.000
    sre_parse.py:139(__setitem__)
    2 0.000 0.000 0.000 0.000
    sre_parse.py:141(__getslice__)
    20 0.001 0.000 0.001 0.000 sre_parse.py:145(append)
    5/3 0.002 0.000 0.002 0.001 sre_parse.py:147(getwidth)
    1 0.000 0.000 0.000 0.000 sre_parse.py:183(__init__)
    25 0.002 0.000 0.002 0.000 sre_parse.py:187(__next)
    3 0.000 0.000 0.000 0.000 sre_parse.py:200(match)
    24 0.003 0.000 0.005 0.000 sre_parse.py:206(get)
    3 0.000 0.000 0.000 0.000 sre_parse.py:269(_escape)
    1 0.000 0.000 0.013 0.013
    sre_parse.py:313(_parse_sub)
    1 0.006 0.006 0.013 0.013 sre_parse.py:368(_parse)
    1 0.000 0.000 0.014 0.014 sre_parse.py:613(parse)
    1 0.000 0.000 0.000 0.000 sre_parse.py:75(__init__)
    3 0.000 0.000 0.000 0.000 sre_parse.py:98(__init__)
    4 0.000 0.000 0.000 0.000 string.py:48(lower)
    profile.run("inspect2.findsource(cgi.FormContent)")
    4105 function calls in 2.441 CPU seconds

    Ordered by: standard name

    ncalls tottime percall cumtime percall filename:lineno(function)
    1 0.023 0.023 2.237 2.237 <string>:1(?)
    1 0.025 0.025 0.025 0.025 inspect2.py:279(getfile)
    1 0.062 0.062 0.176 0.176
    inspect2.py:318(getsourcefile)
    2 0.000 0.000 0.000 0.000 inspect2.py:33(ismodule)
    1 0.834 0.834 2.215 2.215
    inspect2.py:368(findsource)
    2 0.000 0.000 0.000 0.000 inspect2.py:41(isclass)
    1 0.089 0.089 0.089 0.089 ntpath.py:244(exists)
    1 0.203 0.203 2.441 2.441
    profile:0(inspect2.findsource(cgi
    .FormContent))
    0 0.000 0.000 profile:0(profiler)
    4 0.000 0.000 0.000 0.000 string.py:48(lower)
    4091 1.205 0.000 1.205 0.000
    tokenize.py:135(generate_tokens)

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppython-list @
categoriespython
postedJan 10, '02 at 12:15a
activeJan 11, '02 at 8:39p
posts3
users2
websitepython.org

People

Translate

site design / logo © 2022 Grokbase