[Cython] 0.16 release

On Mon, Jan 23, 2012 at 2:27 AM, mark florisson wrote:
Hey,

It's been almost three months since we talked about a 0.16 release, I
think it's quite ready. It would already be a big release, it would be
good to see how people like it, and to catch any issues etc before we
pile on more features.

On 25 January 2012 01:27, Robert Bradshaw wrote:
On Mon, Jan 23, 2012 at 2:27 AM, mark florisson
wrote:
I would love to do a release soon. Last time this came up, I think the
big issue was (compilation) performance regression. Has this been
adequately addressed?

- Robert
_______________________________________________
cython-devel mailing list
cython-devel at python.org
http://mail.python.org/mailman/listinfo/cython-devel

mark florisson, 25.01.2012 11:43:
Sort of. Basically if you don't use memoryviews it will be as fast as
it used to be, otherwise there is about a 3 second constant time
overhead (on my machine).
Oh I see. I suppose to try it out under a debugger one would have to
compile the whole of sage from source?

On 25 January 2012 12:00, Stefan Behnel wrote:
mark florisson, 25.01.2012 11:43:
It might be easier to log into sage.math, go to the sage build directory
that Jenkins uses and do some experiments there. It's in

/levi/scratch/robertwb/hudson/sage-4.8/

Stefan
_______________________________________________
cython-devel mailing list
cython-devel at python.org
http://mail.python.org/mailman/listinfo/cython-devel

On 1/25/12 11:39 AM, Robert Bradshaw wrote:
install
https://sage.math.washington.edu:8091/hudson/view/ext-libs/job/sage-build/lastSuccessfulBuild/artifact/cython-devel.spkg
by downloading it and running "sage -i cython-devel.spkg"

2012/1/26 Jason Grout <jason-sage at creativetrax.com>:


In fact, you could just do

sage -i
https://sage.math.washington.edu:8091/hudson/view/ext-libs/job/sage-build/lastSuccessfulBuild/artifact/cython-devel.spkg

and Sage will (at least, should) download it for you, so that's even one
less step!

Jason

On Sat, Jan 28, 2012 at 8:05 AM, Vitja Makarov wrote:
2012/1/26 Jason Grout <jason-sage at creativetrax.com>:
Thanks for detailed instruction! I've successfully built it.

"sage -t -gdb ./...." doesn't work, is that a bug?

vitja at mchome:~/Downloads/sage-4.8$ ./sage ?-t -gdb
devel/sage/sage/combinat/sf/macdonald.py
sage -t -gdb "devel/sage/sage/combinat/sf/macdonald.py"
********************************************************************************
Type r at the (gdb) prompt to run the doctests.
Type bt if there is a crash to see a traceback.
********************************************************************************
gdb --args python /home/vitja/.sage//tmp/macdonald_6182.py
starting cmd gdb --args python /home/vitja/.sage//tmp/macdonald_6182.py
ImportError: No module named site
? ? ? ? [0.2 s]

----------------------------------------------------------------------
The following tests failed:


? ? ? ?sage -t -gdb "devel/sage/sage/combinat/sf/macdonald.py"
Total time for all tests: 0.2 seconds
Yes, that's a bug.
I've found another way to run tests (using sage -sh and then direct
python ~/.sage/tmp/...py)

So I found one of the problems. Here is minimal cython example:

def foo(values):
? ?return (0,)*len(values)
foo([1,2,3])

len(values) somehow is passed as an integer to PyObject_Multiply()

2012/1/31 Robert Bradshaw <robertwb at math.washington.edu>:
Yeah, that's a bug too :).

I've fixed tuple mult_factor bug here:

https://github.com/cython/cython/commit/2d4b85dbcef885fbdaf6a3b2daef7a017184a56f

On Sat, Feb 4, 2012 at 10:49 AM, Vitja Makarov wrote:
2012/1/31 Robert Bradshaw <robertwb at math.washington.edu>:
I've fixed tuple mult_factor bug here:

https://github.com/cython/cython/commit/2d4b85dbcef885fbdaf6a3b2daef7a017184a56f

No more segfaults in sage-tests but still 7 errors.

Thanks! I've looked into the other errors and I think it boils down to
our use of --disable-function-redefinition being incompatible with how
decorators work. Of course that was just a hack, so I've fixed sage to
build/startup without using that flag, but there's some strangeness
with import order now that I haven't had time to resolve yet.

2012/2/11 Robert Bradshaw <robertwb at math.washington.edu>:
All of Sage passes except for one test:

sage -t ?devel/sage/sage/misc/sageinspect.py
**********************************************************************
File "/levi/scratch/robertwb/hudson/sage-4.8/devel/sage-main/sage/misc/sageinspect.py",
line 970:
? ?sage: sage_getargspec(bernstein_polynomial_factory_ratlist.coeffs_bitsize)
Expected:
? ?ArgSpec(args=['self'], varargs=None, keywords=None, defaults=None)
Got:
? ?ArgSpec(args=['self'], varargs=None, keywords=None, defaults=())
**********************************************************************
File "/levi/scratch/robertwb/hudson/sage-4.8/devel/sage-main/sage/misc/sageinspect.py",
line 973:
? ?sage: sage_getargspec(BooleanMonomialMonoid.gen)
Expected:
? ?ArgSpec(args=['self', 'i'], varargs=None, keywords=None, defaults=(0,))
Got:
? ?ArgSpec(args=['self', 'i'], varargs=None, keywords=None, defaults=())
**********************************************************************
1 items had failures:
? 2 of ?31 in __main__.example_21
***Test Failed*** 2 failures.

Any ideas why this would have changed?

2012/2/12 Vitja Makarov <vitja.makarov at gmail.com>:
2012/2/11 Robert Bradshaw <robertwb at math.washington.edu>:
CyFunction now provides its own code object. So inspect.getargs() is
called instead of
inspect.ArgSpec(*_sage_getargspec_cython(sage_getsource(obj))). It
seems like func.func_defaults should be implemented.

On Sun, Feb 12, 2012 at 12:53 PM, Vitja Makarov wrote:
2012/2/12 Vitja Makarov <vitja.makarov at gmail.com>:
I've created a pull request:

https://github.com/cython/cython/pull/88

On 14 February 2012 07:07, Robert Bradshaw wrote:
Thanks! The only other thing I can think of was a question of using
caching to mitigate the longer compile times, but I can't remember if
this was resolved.

- Robert
_______________________________________________
cython-devel mailing list
cython-devel at python.org
http://mail.python.org/mailman/listinfo/cython-devel

On Tue, Feb 14, 2012 at 7:49 AM, mark florisson wrote:
The compiler has like 2 or 3 seconds of constant overhead if you use
memoryviews.

On 14 February 2012 17:19, Robert Bradshaw wrote:
On Tue, Feb 14, 2012 at 7:49 AM, mark florisson
wrote:
That'd be nice to cut down, but certainly not a blocker.
Twin boys coming any day now!

- Robert
_______________________________________________
cython-devel mailing list
cython-devel at python.org
http://mail.python.org/mailman/listinfo/cython-devel

On Tue, Feb 14, 2012 at 1:09 PM, mark florisson wrote:
And the Cython team just keeps on growing!

On 14 February 2012 21:33, Robert Bradshaw wrote:
On Tue, Feb 14, 2012 at 1:09 PM, mark florisson
wrote:
:)
Thanks!

- Robert
_______________________________________________
cython-devel mailing list
cython-devel at python.org
http://mail.python.org/mailman/listinfo/cython-devel

On 15 February 2012 15:45, mark florisson wrote:
Sorry, my previous email with attachment bounced. Here goes.

I'm getting a substantial amount of failing tests on MSVC,
https://gist.github.com/1836766. I think most complex number tests are
failing because they cast
a struct of a certain type to itself like ((struct_A) my_struct_A),
which MSVC doesn't allow.

Some tests seem to fail because they can't be imported: "compiling (c)
and running numpy_parallel: ImportError: No module named
numpy_parallel".

And then there is a huge number of permission errors: WindowsError:
[Error 5] Access is denied:
'c:\\Users\\mark\\cython\\BUILD\\compile\\cpp\\libc_math.pyd' . Maybe
something is broken in the test runner (or in my setup somehow)?

2012/2/15 mark florisson <markflorisson88 at gmail.com>:
The pasted output is a little munged because it was redirected to a
log (and stdout is probably block buffering, something we could also
fix to line buffering).

On 19 February 2012 10:16, Vitja Makarov wrote:
2012/2/15 mark florisson <markflorisson88 at gmail.com>:
I've merged cydefaults branch and now sage-tests is blue.

--
vitja.
_______________________________________________
cython-devel mailing list
cython-devel at python.org
http://mail.python.org/mailman/listinfo/cython-devel

2012/2/20 mark florisson <markflorisson88 at gmail.com>:
So, if the defaults are literals you build a tuple and set them on the
function, but if they are not literals you save everything in a struct
and use a callback that builds a tuple from the elements of that
struct, correct? Why can't you always just build a tuple, i.e., why do
you need the callback to build the tuple?

2012/2/20 Vitja Makarov <vitja.makarov at gmail.com>:
2012/2/20 mark florisson <markflorisson88 at gmail.com>:
So if defaults are literals const tuple is created once at constant
initialization. ?Since CyFunction.defaults are already there (remember
http://trac.cython.org/cython_trac/ticket/674) I've decided to avoid
defaults tuple initialization at function create time. Instead I've
introduced constructor (defaults_getter) it's run only once and caches
result in CyFunction.defaults_tuple.

ps: We should wait with release until pyregr tests issue is solved.

On 23 February 2012 08:30, Vitja Makarov wrote:
2012/2/20 Vitja Makarov <vitja.makarov at gmail.com>:
We can also fix this ticket before release
http://trac.cython.org/cython_trac/ticket/761

--
vitja.
_______________________________________________
cython-devel mailing list
cython-devel at python.org
http://mail.python.org/mailman/listinfo/cython-devel

2012/2/23 mark florisson <markflorisson88 at gmail.com>:
Good idea. I think the ticket should read 'sys.path' instead of
PYTHONPATH, though.

2012/2/23 Vitja Makarov <vitja.makarov at gmail.com>:
2012/2/23 mark florisson <markflorisson88 at gmail.com>:
Yeah, I think the fix is trivial we should prepend (or append?)
sys.path to cython includes

On 23 February 2012 08:36, Vitja Makarov wrote:
2012/2/23 mark florisson <markflorisson88 at gmail.com>:
Yeah, I think the fix is trivial we should prepend (or append?)
sys.path to cython includes

--
vitja.
_______________________________________________
cython-devel mailing list
cython-devel at python.org
http://mail.python.org/mailman/listinfo/cython-devel

2012/2/23 mark florisson <markflorisson88 at gmail.com>:
I think append, you'd want local things to override installed things
along sys.path.

mark florisson, 23.02.2012 09:38:
I think append, you'd want local things to override installed things
along sys.path.

On 23 February 2012 15:43, Stefan Behnel wrote:
mark florisson, 23.02.2012 09:38:
I think it's

1) user provided directories

2) Cython provided directories

3) sys.path

or would you swap 2) and 3) ?

Stefan
_______________________________________________
cython-devel mailing list
cython-devel at python.org
http://mail.python.org/mailman/listinfo/cython-devel

2012/2/24 S?bastien Sabl? Sabl? <sable at users.sourceforge.net>:
Hi,

could you please also look at incorporating the following patch before
releasing 0.16? (if it has not already been merged)

https://github.com/cython/cython/pull/67

It has been more or less validated, but a test case is needed.

This patch makes using C++ templates much more convenient with Cython.

Currently I have to use hacks like the following which looks ugly and make
the code less readable:
ctypedef TCacheVarData[float] TCacheVarData_float "TCacheVarData<float>"

Also thank you for all the work done on Cython, I have been using it (and
Pyrex before) intensively? for more than 6 years now, and it makes
integrating Python and C/C++ really convenient.

Thanks in advance

S?bastien

_______________________________________________
cython-devel mailing list
cython-devel at python.org
http://mail.python.org/mailman/listinfo/cython-devel

2012/2/24 S?bastien Sabl? Sabl? <sable at users.sourceforge.net>:
Ok I merged it and added a test. I also fixed a lot of tests to run
under MSVC on windows. I'm thinking to merge
https://github.com/cython/cython/pull/77, see if everything still
passes on Jenkins, and then pushing out a beta release for 0.16. I
created some release notes, please feel free to add to the page
(especially to the feature and improvements lists), they might be
incomplete: http://wiki.cython.org/ReleaseNotes-0.16

Are there any other last-minute bug fixes pending?
_______________________________________________
cython-devel mailing list
cython-devel at python.org
http://mail.python.org/mailman/listinfo/cython-devel

2012/2/27 S?bastien Sabl? Sabl? <sable at users.sourceforge.net>:
Great, thanks!


2012/2/25 mark florisson <markflorisson88 at gmail.com>

On 27 March 2012 11:57, Vitja Makarov wrote:
2012/2/27 S?bastien Sabl? Sabl? <sable at users.sourceforge.net>:

Recently I've found this bug on the tracker:

http://trac.cython.org/cython_trac/ticket/766

I think I can fix it before release or wait unitl 0.16.1 since itsn't
a regression.

--
vitja.
_______________________________________________
cython-devel mailing list
cython-devel at python.org
http://mail.python.org/mailman/listinfo/cython-devel

mark florisson, 27.03.2012 13:20:
I tested the release in my own branch and jenkins was blue, but
the release build seems to disagree.

On 27 March 2012 12:58, Stefan Behnel wrote:
mark florisson, 27.03.2012 13:20:
The release (and master) branch is tested against the "-ext" builds of
CPython, which have some external packages installed, including NumPy. You
have to change the config of your "tests" job to use those instead of the
plain CPython builds. The build job can stay as it is.

As for the inner workings, there's a normal "pyXY-hg" job to build CPython
and a corresponding "pyXY-ext-hg" job that takes the build and installs a
list of packages into it, then creates a new install archive. You can then
reference either of the two archives in your build/test jobs by asking for
a "pyXY" or "pyXY-ext" Python.

These are legitimate warnings that are worth fixing (at some point), I
think. They seem to originate from the buffer implementation in numpy.pxd.
Those sort-of-external special methods shouldn't lead to the generation of
a Python wrapper function.

It's surprising that that only occurs in one out of four Py3 test
configurations. Maybe there's something indeterministic in those tests?

Yep, some of them look really funny (complaining about getting exactly the
expected output), others are the typical Py3 problems (e.g. printing bytes).

Note that the reason the py3k tests are not impacted is that it does not
have NumPy. So the tests would equally fail in all Py3 versions.

Stefan
_______________________________________________
cython-devel mailing list
cython-devel at python.org
http://mail.python.org/mailman/listinfo/cython-devel

On Fri, Mar 30, 2012 at 10:24 AM, mark florisson wrote:

The release build looks good now, I'm thinking of pushing a second and
final beta out there tomorrow. If anyone wants to get something in,
now is the time to raise voice.

2012/3/30 Robert Bradshaw <robertwb at gmail.com>:
On Fri, Mar 30, 2012 at 10:24 AM, mark florisson
wrote:
Excellent. I personally don't know of anything that can't wait 'till
0.16.1. If you're confident enough, let's call it a release candidate
:).

mark florisson, 30.03.2012 19:24:
The release build looks good now, I'm thinking of pushing a second and
final beta out there tomorrow. If anyone wants to get something in,
now is the time to raise voice.

On 31 March 2012 10:14, Stefan Behnel wrote:
mark florisson, 30.03.2012 19:24:
Note that Dag's NumPy specialisation broke the Sage build by introducing a
compiler crash.

https://sage.math.washington.edu:8091/hudson/view/cython-devel/job/sage-build/1064/consoleFull

The problem is that the new code is already triggered during type inference
(i.e. before type analysis) and then crashes the compiler because it
accesses types that are not known yet.

Stefan
_______________________________________________
cython-devel mailing list
cython-devel at python.org
http://mail.python.org/mailman/listinfo/cython-devel

mark florisson, 31.03.2012 11:31:
Thanks for pointing that out Stefan. It would probably work if it
passed in 'obj_type' to the numpy_transform_attribute_node, and used
that instead of node.type. Alternatively, it could just wait until the
type is set through analyse_expressions.

On 31 March 2012 17:15, Stefan Behnel wrote:
mark florisson, 31.03.2012 11:31:
Thanks for fixing them. The release branch seems to be in good shape for a
final beta release now.

On a related note, the current master looks more like a 0.17 to me than a
0.16.1 when I compare it to the current release branch. We could just wait
for a couple of weeks to collect bugs in 0.16 from user reports (and
features as they come in), and then push out a 0.17. I hope the pending
PyPy changes will also be ready enough for a release by then, that would
round up the feature list quite nicely.

Stefan
_______________________________________________
cython-devel mailing list
cython-devel at python.org
http://mail.python.org/mailman/listinfo/cython-devel