FAQ

[Cython] PyCon-DE wrap-up by Kay Hayen

Robert Bradshaw
Oct 11, 2011 at 6:11 am
Thanks for the update and link. Sounds like PyCon-DE went well.
On Mon, Oct 10, 2011 at 1:38 AM, Stefan Behnel wrote:
mark florisson, 09.10.2011 19:57:
On 9 October 2011 18:35, Stefan Behnel wrote:

One of the impressions I took out of the technical discussions with Kay
is
that there isn't really a good reason why Cython should refuse to
duplicate
some of the inner mechanics of CPython for optimisation purposes. Nuitka
appears to be somewhat more aggressive here, partly because Kay doesn't
currently care all that much about portability (e.g. to Python 3).
Interesting. What kind of (significant) optimizations could be made by
duplicating code? Do you want to duplicate entire functions or do you
want to inline parts of those?
I was mainly referring to things like direct access to type/object struct
fields and little things like that. They can make a difference especially in
loops, compared to calling into a generic C-API function. For example, we
could have our own interned implementation of PyDict_Next(). I'm not very
impressed by the performance of that C-API function - repeated calls to
GetItem can be faster than looping over a dict with PyDict_Next()!

That being said, I wasn't referring to any specific changes. It was more of
a general remark about the invisible line that we currently draw in Cython.
CPython, especially the internals, is a slow enough moving target that
I'm not too concerned about reaching into the internals if there is a
clear benefit. If we're flexible enough to support 2.x and 3.x, I
think we can handle 3.(x+1) when it comes.
I actually think we should not get too tied to CPython, e.g. what if
PyPy gets a CPython compatible API
It already implements a part of the C-API:

http://morepypy.blogspot.com/2010/04/using-cpython-extension-modules-with.html

However, if we really want to support it at that level, there's likely more
to do than just removing low-level optimisations. And that would take the
normal route that we always use: macros and conditionally compiled inline
functions. The mere fact that we try to support different targets doesn't
mean that we should stop optimising for specific targets. +1
The same is true
for different versions of CPython, where we often use better optimisations
in newer releases, without sacrificing backwards compatibility.

Personally, I think that supporting PyPy at the Python level is a lot more
interesting, although it may be easier to get it working at the cpyext
level.

or possibly a subset like PEP 384?
That's currently not very interesting since there are basically no C
extensions around (generated or hand written) that restrict themselves to
that API. Supporting it in Cython would mean that we have to rewrite huge
parts of the generated C code. It's not even clear to me yet that we *can*
implement all of Cython's features based on PEP 384. For example, fast
indexing into lists and tuples is basically a no-no in the restricted C-API.
There are tons of rather unexpected restrictions like this.
I agree, PEP 384 is a nice idea, but it seems to be a rather lot of
work for an unclear/small benefit (compared to other stuff we could be
doing.)

- Robert
reply

Search Discussions

Discussion Posts

Previous

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 5 of 6 | next ›