[Python] list comprehension question

The biggest thing to learn about list comprehensions is when not to use
them. I can't imagine how your latter version (even if correct) is
clearer than the first.

I think he might be using the wrong function for a matrix
multiplication, not that it's not workable, but there are other
libraries like numpy that could help out.


I wouldn't use list comprehension for this, unless it might be several
lists that interact.

Try this in google:


python matrix multiplication

Is it not true that list comprehension is much faster the the for loops?


If it is not the correct way of doing this, i appoligize.
Like i said, I'm learing list comprehension.


Thanks
Kevin
On Oct 16, 2012 10:14 PM, "Dave Angel" wrote:

I thought it was matrix multiplication mixed with list comprehension.


Check this out real quick from the docs for list comprehension, but if
it's a mixture of matrix multi, and list comp, then reply back.


http://docs.python.org/tutorial/datastructures.html#list-comprehensions

And here is two more to read up on:


http://www.scipy.org/FAQ#head-045cba7978d75ec548822222150fa6ad308e325a


www.python-course.eu/matrix_arithmetic.php

If you're looking for a fast solution then you should follow David's
suggestion and use numpy:


The most relevant numpy function is numpy.dot:
http://docs.scipy.org/doc/numpy/reference/generated/numpy.dot.html




Oscar


P.S. David, it was a bit fiddly for me to quote your response because
the relevant part was in a post that had no context. Also your name is
listed as Dwight above. That is automatically added by gmail because
of the settings in your own email client. You may need to change your
account settings if you want it to say David when people quote you.

(Please don't top-post; it ruins the ordering. In these forums, put
your response after the part you quote from earlier messages. Or even
better, after each part you quote. Then trim off the parts you didn't
reference.)


list comprehensions CAN be much faster, but not necessarily. The most
complex a loop, the less likely it'll help much.


In any case, only the inner loop will be affected. Nesting two list
comprehensions will make a trivial difference.


On the other hand, Hans Mulder shows some other factoring which seems
much more readable than yours.


Studying (and testing) those could teach you a lot about comprehensions,
as well as about the libraries that can help. Note especially what
zip(*b) yields, and think about what it means.

One-lining the comprehension seems to make a difference of about 10%
out here. Maybe Ive missed something? Seems too large?


# My original suggestion
def dot(p,q): return sum (x*y for x,y in zip(p,q))
def transpose(m): return zip(*m)
def mm(a,b): return mmt(a, transpose(b))
def mmt(a,b): return [[dot(ra, rb) for rb in b] for ra in a]


# One-liner (Thanks Hans for reminding me of sum)


def mm1(a,b): return [[sum([x*y for x,y in zip(ra,rb)]) for rb in
zip(*b)] for ra in a]

12.276363849639893
13.453603029251099

In case anyone wants to try out with the same data, I used:


m = [range(i,i+30) for i in range(30)]

And I'd wager all the improvement is in the inner loop, the dot() function.

Dave Angel? 2012?10?17????UTC+8??10?37?01????

Thanks for the tips of matrix operations over some fields or rings
other than the real field and the complex field.

Sorry -- red herring!


Changing


def mm1(a,b): return [[sum(x*y for x,y in zip(ra,rb)) for rb in
zip(*b)] for ra in a]


to


def mm1(a,b): return [[sum([x*y for x,y in zip(ra,rb)]) for rb in
zip(*b)] for ra in a]


makes the speed diff vanish

rusi? 2012?10?17????UTC+8??10?50?11????

I think a lot people don't work on computations over
fields other real and complex.


That is why a lot people keep complaining about the speeds
of python programs executed.

I know this doesn't answer the python question, but it will help you
algorithm out what you need to know.


I do lots of interdisciplinary research, so if this doesn't help, let
me know, and I'll take a refresher, and work up some code. Also, look
at numpy.

Try rewriting using dot


from operator import mul
def dot(p,q):
return reduce(mul, [x*y for x,y in zip(p,q)]) # the [] can become
()


and avoiding object-orientation (at least to start with)

list comprehensions specifically abbreviate the code that they are
(essentially) equivalent to.


res = []
for item in source:
res.append(f(item))
res


<==>


[f(item) for item in source]


Matrix multiplication does not fit the pattern above. The reduction is
number addition rather than list appending.

Dunno why you say that. Heres matrix multiply using list
comprehensions:


from operator import add
def dot(p,q): return reduce(add, (x*y for x,y in zip(p,q)))


def transpose(m): return zip(*m)


def mm(a,b): return mmt(a, transpose(b))


def mmt(a,b): return [[dot(ra, rb) for rb in b] for ra in a]


which can then be 'reduced' to a one-liner if that takes your fancy

I can golf it down to two lines without losing readability:


def dot(p,q): return sum(x*y for x,y in zip(p,q))


def mm(a,b): return [[dot(ra, rb) for rb in zip(*b)] for ra in a]




Hope this helps,


-- HansM

My response is to the part Kevin could *not* convert, not the parts he
did convert. I attempted to explain why he could not convert that part.


Because it is true and because it makes an essential point about what
one can and cannot sensibly do with comprehensions. They are not
intended to be a replacement for *all* loops.


The essential inner reduction by addition of products that Kevin was
'not able to convert' cannot be converted (with out some obnoxious
trickery and *some* extra helper), so his request for a sensible
conversion is futile.


plus a helper function that does the inner reduction otherwise, as I
implied it should be


Right, this is the addition reduction that the OP was trying to
convert to a list comp. It cannot be done and you have not done it
either. Note the the vector of products is produced as a comprehension.
That you left it as a 'generator expression' is not relevant.


The important point is the the addition combines the products of
different iterations and list comps, by their nature, cannot directly do
that.


This is the repeated append part of the original nested loops and that,
as I said, can be re-expressed as a list comp. But that was not the part
Kevin was having difficulty with and not the part I was talking about.