ID: 18039
Updated by: gsherwood@squiz.net
Reported By: oliver dot klee at diemedialen dot de
Summary: JS Tokenizer crash when ] is last character in file
-Status: Open
+Status: Feedback
Type: Bug
Package: PHP_CodeSniffer
Operating System: Kubuntu Linux 10.10
Package Version: 1.3.0RC1
PHP Version: 5.3.3
Roadmap Versions:
New Comment:
-Status: Open
+Status: Feedback
I can't replicate this issue. Can you please use the -vv command line
argument to generate the tokenizer output. It should look like this:
94:trunk gsherwood$ php scripts/phpcs --standard=Generic temp.js -vv
Registering sniffs in Generic standard... DONE (36 sniffs registered)
Processing temp.js
*** START TOKENIZING ***
Process char 0 => [ (buffer: )
* char is token, looking ahead 8 chars *
=> Looking ahead 1 chars => []
=> Looking ahead 2 chars => []\n
* look ahead found nothing *
=> Added token T_OPEN_SQUARE_BRACKET ([)
Process char 1 => ] (buffer: )
* buffer possibly contains token, looking ahead 8 chars *
=> Looking ahead 1 chars => ]\n
* look ahead found nothing *
=> Added token T_CLOSE_SQUARE_BRACKET (])
Process char 2 => \n (buffer: )
*** END TOKENIZING ***
*** START BRACKET MAP ***
=> Found square bracket opener at 1
=> Found square bracket closer at 2 for 1
*** END BRACKET MAP ***
*** START SCOPE MAP ***
*** END SCOPE MAP ***
*** START LEVEL MAP ***
Process token 0 on line 1 [lvl:0;]: T_OPEN_TAG =>
Process token 1 on line 1 [lvl:0;]: T_OPEN_SQUARE_BRACKET => [
Process token 2 on line 1 [lvl:0;]: T_CLOSE_SQUARE_BRACKET => ]
Process token 3 on line 1 [lvl:0;]: T_CLOSE_TAG =>
*** END LEVEL MAP ***
*** START ADDITIONAL JS PROCESSING ***
Process token 0: T_OPEN_TAG =>
Process token 1: T_OPEN_SQUARE_BRACKET => [
Process token 2: T_CLOSE_SQUARE_BRACKET => ]
Process token 3: T_CLOSE_TAG =>
*** END ADDITIONAL JS PROCESSING ***
[4 tokens in 1 lines]...
DONE in < 1 second (0 errors, 0 warnings)
Time: 0 seconds, Memory: 4.25Mb
But I'll be looking for any differences of course.
Previous Comments:
------------------------------------------------------------------------
[2010-11-09 18:52:06] oliverklee
Description:
------------
1. Create a JS file "test.js" with just the contents "[]" (without the
quotes, without any whitespace).
2. Check that file using any Standard, e.g. the Generic standard
Test script:
---------------
[]
Expected result:
----------------
Only the time and memory get displayed.
Actual result:
--------------
klee@dmok:~$ phpcs --standard=Generic /tmp/test.js
PHP Notice: Undefined offset: 2 in
/usr/share/php/PHP/CodeSniffer/Tokenizers/JS.php on line
444
Time: 0 seconds, Memory: 2.25Mb
This is the offending line:
if (in_array(strtolower($buffer), $tokenTypes) ===
true
&& (preg_match('|[a-zA-z0-9_]|', $char) === 0
preg_match('|[a-zA-z0-9_]|', $chars[($i + 1)])
The $i + 1 is the offset that is too big.
------------------------------------------------------------------------
