[Python] Questions: While And List Comprehension

On 11/10/2010 4:36 AM Felipe Vinturini said...
Easiest would be print [ v for v in sys.stdin.readlines()[:5] ] but that
still reads the entire sys.stdin (whatever it may be...)
Well, you could use

def X(iNumber):
print sum([iNumber/(5**ii) for ii in range(1,2*int(len("%s" %
iNumber)))])

but the range selection is rather arbitrary.

Emile

Here's a way of doing the same thing without consuming the entire
stream (sys.stdin):

#!/usr/bin/env python

import sys
print [v for v in list(line for line in sys.stdin)[:5]]

This uses a generator expression to read from stdin, converts this to
a list (only getting the first 5 items).

cheers
James

'list' will exhaust the input, then the slicing will return at most 5
lines.

Also, the list comprehension and generator expression are both the
identity versions thereof, so a shorter equivalent version of the
erroneous code would be simply:
print list(sys.stdin)[:5]

Cheers,
Chris

Hmm you're right :)

A generator expression slicing only accesses as many elements as it
needs to? But surely list(line for line in sys.stdin) will be evaluated
before the indexing is applied?

I'd have thought that would have exhausted the input file.
... f.write("%s\n" % i)
...
['one\n', 'two\n', 'three\n', 'four\n', 'five\n']
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
StopIteration
>>>

This suggests that you are mistaken about not exhausting the source.

regards
Steve

Yeah I was mistaken. Oh well :) I was thinking of a generator-based
solution and got lost in the implementation!

print [v for v in sys.stdin.readlines()[:5]]

cheers
James

how about print [sys.stdin.readline() for i in range(5)]

At least that won't consume the whole file.

regards
Steve

+1 on this approach. Clear and obvious and not reliant on any library
modules other than sys.

itertools, what WAS I thinking? :)

maybe:

import sys
from itertools import islice

print [v for v in islice((line for line in sys.stdin), 0, 5)]

?

(line for line in sys.stdin)

and sys.stdin are equivalent as are

[v for v in items]

and list(items)

So

print list(islice(sys.stdin, 5))

Peter

Hello Folks,

You are great! Thanks for the suggestions!

With the values between # #, I meant I would like to limit the values
sys.stdin.readlines() read (but I think there is no way to do it); instead
of reading the entire input, limit it to read only 5 entries.
I knew about readline(), but when checking it with cProfile each readline()
needs a function call and with readlines() it is only one. I think that
depending on input, readline() calls may have a higher cost.

Check below:

My test file: TestFile.txt
=====================================================
1
2
3
4
5
6
7
8
9
10
=====================================================

My test code: simpleReadLines.py
=====================================================
import sys
import cProfile

def simpleTestLimitRLS(tLimit):
sysStdinReadLines = sys.stdin.readlines
allValues = [ int(v) for v in sysStdinReadLines() ]
print allValues[:tLimit]

def simpleTestLimitRL(tLimit):
sysStdinReadLine = sys.stdin.readline
print [ sysStdinReadLine() for x in xrange(0,5) ]

if __name__ == '__main__':
cProfile.run("simpleTestLimitRL(5)")
cProfile.run("simpleTestLimitRLS(5)")
=====================================================

Test result:
=====================================================
['1\n', '2\n', '3\n', '4\n', '5\n']
8 function calls in 0.004 CPU seconds

Ordered by: standard name

ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 0.004 0.004 <string>:1(<module>)
1 0.004 0.004 0.004 0.004
simpleReadLines.py:13(simpleTestLimitRL)
1 0.000 0.000 0.000 0.000 {method 'disable' of
'_lsprof.Profiler' objects}
5 0.000 0.000 0.000 0.000 {method 'readline' of 'file'
objects}

[6, 7, 8, 9, 10]
4 function calls in 0.003 CPU seconds

Ordered by: standard name

ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 0.003 0.003 <string>:1(<module>)
1 0.003 0.003 0.003 0.003
simpleReadLines.py:8(simpleTestLimitRLS)
1 0.000 0.000 0.000 0.000 {method 'disable' of
'_lsprof.Profiler' objects}
1 0.000 0.000 0.000 0.000 {method 'readlines' of 'file'
objects}
=====================================================

If I change the execution order to:
1. simpleTestLimitRLS(5)
2. simpleTestLimitRL(5)

Of course, python returns an error because all the input has been consumed
by readlines().

Again, thanks for your attention and suggestions.

Regards,
Felipe.

I was being a little verbose ... But I like your simplification :)

cheers
James

On 11/10/2010 4:36 AM Felipe Vinturini said...
Probably not in a way that's more efficient than what you're already
doing. One thing you should do is to use the // operator instead
of /. Apart from being more portable (/ no longer does integer
division in Python 3), it's also significantly faster:

$ python -m timeit "12 / 5"
10000000 loops, best of 3: 0.0711 usec per loop

$ python -m timeit "12 // 5"
10000000 loops, best of 3: 0.028 usec per loop

Also, don't use a list comprehension for your looping construct in
factorialCountZeros2(). The list comprehension has to build up a list
containing all the None objects returned by the Z() method, which
wastes time and space. Because of that, it would be faster just to
write it out the for loop. Besides the speed improvement, it's also
bad style to use a list comprehension for a loop where you're only
interested in the side effects.

Another thing to consider is that function calls are expensive. Your
Z() function gets called once for every line of input, which could add
up. While it will make the code a bit less readable (CodeChef is
definitely not the place to go if you want to learn to write *clear*
code), you might see some improvement if you wrap that function into
the factorialCountZeros2() function.

Finally, one thing that really tends to help a lot in these puzzles is
to do all your input at once and all your output at once. I/O is
crazy expensive, but you can cut a lot of that out by minimizing the
number of reads and writes that you have to do.

After the first readline, you can do something like "input =
sys.stdin.read().strip().split('\n')" to pull everything into a single
list, then iterate over the list instead of stdin. For output, append
all your results to a list instead of printing them immediately, then
use '\n'.join() to generate something suitable for printing.

Cheers,
Ian

P.S. CodeChef and SPOJ both have the psyco module available. If you
just can't seem to get a problem solved in the time limit, you can
always turn it on for a major speed boost. It's more fun to try to
solve the problems without it, though.

Hey all you guys could be mistaken. It looks like the OP is trying to
implement some sort of ghetto database. And he may NOT necessarily
want the FIRST five lines of the file. He said (and i quote) "the
values /between/ # and #". Where # and # are starting and ending of a
range -- not necessarily the starting of the input file! However we
really need the OP to clear up these questions!

Valid point Rick. And yes my earlier post was very wrong :/

My follow-up (after realizing my mistake) might be better (using islice).

cheers
James

Use readline() + a counter instead...

print [sys.stdin.readline() for x in xrange(0,5)]

Roger