[Python] SyntaxError not honoured in list comprehension?

Why? If Python itself has no problem parsing this code, why should it
be so difficult for editors? Python's grammar is fairly simple: it's
LL(1) (unlike C's, for example), so can be parsed with only 1 token of
lookahead.

Bah. Ignore the bit about C. I was thinking that the dangling else
issue made this a problem, but now I'm not sure that's true.

It does seem strange that Python's lexer wouldn't consider 9for as a
single token. Even tough it's not a valid token in Python, your eye
kind of sees it as one, so wouldn't it be better to raise a syntax
error?

Some other places were keyword can follow a number:

9if 0 else 1 (but not "9if 0else 1")
9and 0
9or 0
9in (1,2,3)
9is None

I'm not sure why an odd corner of the grammar would mess the whole
thing up. Most code stylers only approximate the actual grammar
anyway.


Carl Banks

Thanks for having explained in good English my feelings.

Note, that this does not envolve numbers only.
['z', 'z', 'z']
z
>>>


I guess, most editors (so do I) are mainly using
a "re" engine for their styling.

---

Not a keyword, but space related, what should I thing
about this?
Traceback (most recent call last):
File "<psi last command>", line 1, in <module>
NameError: name 'print9' is not defined
9
abc
File "<psi last command>", line 1
print9.0
^
SyntaxError: invalid syntax
>>>

Regards,
jmf

looks like an identifier
can't be a single identifier. Maybe it's a print statement followed by
stuff? (stop being a statement, print!)
can't be an identifier or string literal. Maybe it's a print statement
followed by stuff?
looks like getattr(print9, '0') - but '0' is not a valid name.
Impossible. Error!
somewhat strange, yes.

There are two tokens, "print9" (a name) and ".0" (a float constant) --
looks like SyntaxError to me.

Yep. Looks that way to me, too.

Python 2.7.0+ (release27-maint:82569, Jul 5 2010, 08:35:08)
[GCC 4.0.1 (Apple Inc. build 5490)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
... print token.tok_name[tok[0]], tok[1]
...
NAME print9
NUMBER .0
ENDMARKER

Thank you all for the discussion and the explanations.
I toyed a littled bit this afternoon and I wrote a colouriser
(British spelling?) with the tokenize module. It is quite
simple and easy.

BTW, if I understand correctly the module tokenize import
the module token. So your example becomes:
print tokenize.tok_name[tok[0]], tok[1]

NAME print9
NUMBER .0
ENDMARKER
>>>

Ah yes; you're right. Thanks!

Mark