FAQ
Edit report at http://pear.php.net/bugs/bug.php?id=18039&edit=1

ID: 18039
Updated by: gsherwood@squiz.net
Reported By: oliver dot klee at diemedialen dot de
Summary: JS Tokenizer crash when ] is last character in file
-Status: Open
+Status: Feedback
Type: Bug
Package: PHP_CodeSniffer
Operating System: Kubuntu Linux 10.10
Package Version: 1.3.0RC1
PHP Version: 5.3.3
Roadmap Versions:
New Comment:

-Status: Open
+Status: Feedback
I can't replicate this issue. Can you please use the -vv command line
argument to generate the tokenizer output. It should look like this:

94:trunk gsherwood$ php scripts/phpcs --standard=Generic temp.js -vv
Registering sniffs in Generic standard... DONE (36 sniffs registered)
Processing temp.js
*** START TOKENIZING ***
Process char 0 => [ (buffer: )
* char is token, looking ahead 8 chars *
=> Looking ahead 1 chars => []
=> Looking ahead 2 chars => []\n
* look ahead found nothing *
=> Added token T_OPEN_SQUARE_BRACKET ([)
Process char 1 => ] (buffer: )
* buffer possibly contains token, looking ahead 8 chars *
=> Looking ahead 1 chars => ]\n
* look ahead found nothing *
=> Added token T_CLOSE_SQUARE_BRACKET (])
Process char 2 => \n (buffer: )
*** END TOKENIZING ***
*** START BRACKET MAP ***
=> Found square bracket opener at 1
=> Found square bracket closer at 2 for 1
*** END BRACKET MAP ***
*** START SCOPE MAP ***
*** END SCOPE MAP ***
*** START LEVEL MAP ***
Process token 0 on line 1 [lvl:0;]: T_OPEN_TAG =>
Process token 1 on line 1 [lvl:0;]: T_OPEN_SQUARE_BRACKET => [
Process token 2 on line 1 [lvl:0;]: T_CLOSE_SQUARE_BRACKET => ]
Process token 3 on line 1 [lvl:0;]: T_CLOSE_TAG =>
*** END LEVEL MAP ***
*** START ADDITIONAL JS PROCESSING ***
Process token 0: T_OPEN_TAG =>
Process token 1: T_OPEN_SQUARE_BRACKET => [
Process token 2: T_CLOSE_SQUARE_BRACKET => ]
Process token 3: T_CLOSE_TAG =>
*** END ADDITIONAL JS PROCESSING ***
[4 tokens in 1 lines]...
DONE in < 1 second (0 errors, 0 warnings)
Time: 0 seconds, Memory: 4.25Mb

But I'll be looking for any differences of course.


Previous Comments:
------------------------------------------------------------------------

[2010-11-09 18:52:06] oliverklee

Description:
------------
1. Create a JS file "test.js" with just the contents "[]" (without the
quotes, without any whitespace).
2. Check that file using any Standard, e.g. the Generic standard

Test script:
---------------
[]

Expected result:
----------------
Only the time and memory get displayed.

Actual result:
--------------
klee@dmok:~$ phpcs --standard=Generic /tmp/test.js
PHP Notice: Undefined offset: 2 in
/usr/share/php/PHP/CodeSniffer/Tokenizers/JS.php on line
444
Time: 0 seconds, Memory: 2.25Mb

This is the offending line:


if (in_array(strtolower($buffer), $tokenTypes) ===
true
&& (preg_match('|[a-zA-z0-9_]|', $char) === 0
preg_match('|[a-zA-z0-9_]|', $chars[($i + 1)])
=== 0)

The $i + 1 is the offset that is too big.

------------------------------------------------------------------------

Search Discussions

  • Gsherwood at Nov 10, 2010 at 12:01 am
    Edit report at http://pear.php.net/bugs/bug.php?id=18039&edit=1

    ID: 18039
    Updated by: gsherwood@squiz.net
    Reported By: oliver dot klee at diemedialen dot de
    Summary: JS Tokenizer crash when ] is last character in file
    Status: Feedback
    Type: Bug
    Package: PHP_CodeSniffer
    Operating System: Kubuntu Linux 10.10
    Package Version: 1.3.0RC1
    PHP Version: 5.3.3
    -Assigned To:
    +Assigned To: squiz
    Roadmap Versions:
    New Comment:

    -Assigned To:
    +Assigned To: squiz



    Previous Comments:
    ------------------------------------------------------------------------

    [2010-11-10 01:01:56] squiz

    -Status: Open
    +Status: Feedback
    I can't replicate this issue. Can you please use the -vv command line
    argument to generate the tokenizer output. It should look like this:

    94:trunk gsherwood$ php scripts/phpcs --standard=Generic temp.js -vv
    Registering sniffs in Generic standard... DONE (36 sniffs registered)
    Processing temp.js
    *** START TOKENIZING ***
    Process char 0 => [ (buffer: )
    * char is token, looking ahead 8 chars *
    => Looking ahead 1 chars => []
    => Looking ahead 2 chars => []\n
    * look ahead found nothing *
    => Added token T_OPEN_SQUARE_BRACKET ([)
    Process char 1 => ] (buffer: )
    * buffer possibly contains token, looking ahead 8 chars *
    => Looking ahead 1 chars => ]\n
    * look ahead found nothing *
    => Added token T_CLOSE_SQUARE_BRACKET (])
    Process char 2 => \n (buffer: )
    *** END TOKENIZING ***
    *** START BRACKET MAP ***
    => Found square bracket opener at 1
    => Found square bracket closer at 2 for 1
    *** END BRACKET MAP ***
    *** START SCOPE MAP ***
    *** END SCOPE MAP ***
    *** START LEVEL MAP ***
    Process token 0 on line 1 [lvl:0;]: T_OPEN_TAG =>
    Process token 1 on line 1 [lvl:0;]: T_OPEN_SQUARE_BRACKET => [
    Process token 2 on line 1 [lvl:0;]: T_CLOSE_SQUARE_BRACKET => ]
    Process token 3 on line 1 [lvl:0;]: T_CLOSE_TAG =>
    *** END LEVEL MAP ***
    *** START ADDITIONAL JS PROCESSING ***
    Process token 0: T_OPEN_TAG =>
    Process token 1: T_OPEN_SQUARE_BRACKET => [
    Process token 2: T_CLOSE_SQUARE_BRACKET => ]
    Process token 3: T_CLOSE_TAG =>
    *** END ADDITIONAL JS PROCESSING ***
    [4 tokens in 1 lines]...
    DONE in < 1 second (0 errors, 0 warnings)
    Time: 0 seconds, Memory: 4.25Mb

    But I'll be looking for any differences of course.

    ------------------------------------------------------------------------

    [2010-11-09 18:52:06] oliverklee

    Description:
    ------------
    1. Create a JS file "test.js" with just the contents "[]" (without the
    quotes, without any whitespace).
    2. Check that file using any Standard, e.g. the Generic standard

    Test script:
    ---------------
    []

    Expected result:
    ----------------
    Only the time and memory get displayed.

    Actual result:
    --------------
    klee@dmok:~$ phpcs --standard=Generic /tmp/test.js
    PHP Notice: Undefined offset: 2 in
    /usr/share/php/PHP/CodeSniffer/Tokenizers/JS.php on line
    444
    Time: 0 seconds, Memory: 2.25Mb

    This is the offending line:


    if (in_array(strtolower($buffer), $tokenTypes) ===
    true
    && (preg_match('|[a-zA-z0-9_]|', $char) === 0
    preg_match('|[a-zA-z0-9_]|', $chars[($i + 1)])
    === 0)

    The $i + 1 is the offset that is too big.

    ------------------------------------------------------------------------
  • Oliver Klee at Nov 10, 2010 at 10:02 am
    Edit report at http://pear.php.net/bugs/bug.php?id=18039&edit=1

    ID: 18039
    Updated by: oliver.klee@diemedialen.de
    Reported By: oliver dot klee at diemedialen dot de
    Summary: JS Tokenizer crash when ] is last character in file
    Status: Feedback
    Type: Bug
    Package: PHP_CodeSniffer
    Operating System: Kubuntu Linux 10.10
    Package Version: 1.3.0RC1
    PHP Version: 5.3.3
    Assigned To: squiz
    Roadmap Versions:
    New Comment:

    klee@dmok:~$ phpcs --standard=Generic -vv Downloads/test.js
    Registering sniffs in Generic standard... DONE (35 sniffs registered)
    Processing test.js
    *** START TOKENIZING ***
    Process char 0 => [ (buffer: )
    * char is token, looking ahead 8 chars *
    => Looking ahead 1 chars => []
    * look ahead found nothing *
    => Added token T_OPEN_SQUARE_BRACKET ([)
    Process char 1 => ] (buffer: )
    PHP Notice: Undefined offset: 2 in
    /usr/share/php/PHP/CodeSniffer/Tokenizers/JS.php on line 444
    * buffer possibly contains token, looking ahead 8 chars *
    * look ahead found nothing *
    => Added token T_CLOSE_SQUARE_BRACKET (])
    *** END TOKENIZING ***


    *** START BRACKET MAP ***


    => Found square bracket opener at 1


    => Found square bracket closer at 2 for 1


    *** END BRACKET MAP ***


    *** START SCOPE MAP ***


    *** END SCOPE MAP ***


    *** START LEVEL MAP ***


    Process token 0 on line 1 [lvl:0;]: T_OPEN_TAG =>


    Process token 1 on line 1 [lvl:0;]: T_OPEN_SQUARE_BRACKET => [


    Process token 2 on line 1 [lvl:0;]: T_CLOSE_SQUARE_BRACKET => ]


    Process token 3 on line 1 [lvl:0;]: T_CLOSE_TAG =>


    *** END LEVEL MAP ***
    *** START ADDITIONAL JS PROCESSING ***
    Process token 0: T_OPEN_TAG =>
    Process token 1: T_OPEN_SQUARE_BRACKET => [
    Process token 2: T_CLOSE_SQUARE_BRACKET => ]
    Process token 3: T_CLOSE_TAG =>
    *** END ADDITIONAL JS PROCESSING ***
    [4 tokens in 1 lines]...
    DONE in < 1 second (0 errors, 0 warnings)
    Time: 0 seconds, Memory: 2.25Mb


    Previous Comments:
    ------------------------------------------------------------------------

    [2010-11-10 01:02:03] squiz

    -Assigned To:
    +Assigned To: squiz


    ------------------------------------------------------------------------

    [2010-11-10 01:01:56] squiz

    -Status: Open
    +Status: Feedback
    I can't replicate this issue. Can you please use the -vv command line
    argument to generate the tokenizer output. It should look like this:

    94:trunk gsherwood$ php scripts/phpcs --standard=Generic temp.js -vv
    Registering sniffs in Generic standard... DONE (36 sniffs registered)
    Processing temp.js
    *** START TOKENIZING ***
    Process char 0 => [ (buffer: )
    * char is token, looking ahead 8 chars *
    => Looking ahead 1 chars => []
    => Looking ahead 2 chars => []\n
    * look ahead found nothing *
    => Added token T_OPEN_SQUARE_BRACKET ([)
    Process char 1 => ] (buffer: )
    * buffer possibly contains token, looking ahead 8 chars *
    => Looking ahead 1 chars => ]\n
    * look ahead found nothing *
    => Added token T_CLOSE_SQUARE_BRACKET (])
    Process char 2 => \n (buffer: )
    *** END TOKENIZING ***
    *** START BRACKET MAP ***
    => Found square bracket opener at 1
    => Found square bracket closer at 2 for 1
    *** END BRACKET MAP ***
    *** START SCOPE MAP ***
    *** END SCOPE MAP ***
    *** START LEVEL MAP ***
    Process token 0 on line 1 [lvl:0;]: T_OPEN_TAG =>
    Process token 1 on line 1 [lvl:0;]: T_OPEN_SQUARE_BRACKET => [
    Process token 2 on line 1 [lvl:0;]: T_CLOSE_SQUARE_BRACKET => ]
    Process token 3 on line 1 [lvl:0;]: T_CLOSE_TAG =>
    *** END LEVEL MAP ***
    *** START ADDITIONAL JS PROCESSING ***
    Process token 0: T_OPEN_TAG =>
    Process token 1: T_OPEN_SQUARE_BRACKET => [
    Process token 2: T_CLOSE_SQUARE_BRACKET => ]
    Process token 3: T_CLOSE_TAG =>
    *** END ADDITIONAL JS PROCESSING ***
    [4 tokens in 1 lines]...
    DONE in < 1 second (0 errors, 0 warnings)
    Time: 0 seconds, Memory: 4.25Mb

    But I'll be looking for any differences of course.

    ------------------------------------------------------------------------

    [2010-11-09 18:52:06] oliverklee

    Description:
    ------------
    1. Create a JS file "test.js" with just the contents "[]" (without the
    quotes, without any whitespace).
    2. Check that file using any Standard, e.g. the Generic standard

    Test script:
    ---------------
    []

    Expected result:
    ----------------
    Only the time and memory get displayed.

    Actual result:
    --------------
    klee@dmok:~$ phpcs --standard=Generic /tmp/test.js
    PHP Notice: Undefined offset: 2 in
    /usr/share/php/PHP/CodeSniffer/Tokenizers/JS.php on line
    444
    Time: 0 seconds, Memory: 2.25Mb

    This is the offending line:


    if (in_array(strtolower($buffer), $tokenTypes) ===
    true
    && (preg_match('|[a-zA-z0-9_]|', $char) === 0
    preg_match('|[a-zA-z0-9_]|', $chars[($i + 1)])
    === 0)

    The $i + 1 is the offset that is too big.

    ------------------------------------------------------------------------
  • Gsherwood at Nov 11, 2010 at 11:22 pm
    Edit report at http://pear.php.net/bugs/bug.php?id=18039&edit=1

    ID: 18039
    Updated by: gsherwood@squiz.net
    Reported By: oliver dot klee at diemedialen dot de
    Summary: JS Tokenizer crash when ] is last character in file
    -Status: Feedback
    +Status: Closed
    Type: Bug
    Package: PHP_CodeSniffer
    Operating System: Kubuntu Linux 10.10
    Package Version: 1.3.0RC1
    PHP Version: 5.3.3
    Assigned To: squiz
    Roadmap Versions:
    New Comment:

    -Status: Feedback
    +Status: Closed
    This bug has been fixed in SVN.

    If this was a documentation problem, the fix will appear on pear.php.net
    by the end of next Sunday (CET).

    If this was a problem with the pear.php.net website, the change should
    be live shortly.

    Otherwise, the fix will appear in the package's next release.

    Thank you for the report and for helping us make PEAR better.




    Previous Comments:
    ------------------------------------------------------------------------

    [2010-11-10 11:02:59] oliverklee

    klee@dmok:~$ phpcs --standard=Generic -vv Downloads/test.js
    Registering sniffs in Generic standard... DONE (35 sniffs registered)
    Processing test.js
    *** START TOKENIZING ***
    Process char 0 => [ (buffer: )
    * char is token, looking ahead 8 chars *
    => Looking ahead 1 chars => []
    * look ahead found nothing *
    => Added token T_OPEN_SQUARE_BRACKET ([)
    Process char 1 => ] (buffer: )
    PHP Notice: Undefined offset: 2 in
    /usr/share/php/PHP/CodeSniffer/Tokenizers/JS.php on line 444
    * buffer possibly contains token, looking ahead 8 chars *
    * look ahead found nothing *
    => Added token T_CLOSE_SQUARE_BRACKET (])
    *** END TOKENIZING ***


    *** START BRACKET MAP ***


    => Found square bracket opener at 1


    => Found square bracket closer at 2 for 1


    *** END BRACKET MAP ***


    *** START SCOPE MAP ***


    *** END SCOPE MAP ***


    *** START LEVEL MAP ***


    Process token 0 on line 1 [lvl:0;]: T_OPEN_TAG =>


    Process token 1 on line 1 [lvl:0;]: T_OPEN_SQUARE_BRACKET => [


    Process token 2 on line 1 [lvl:0;]: T_CLOSE_SQUARE_BRACKET => ]


    Process token 3 on line 1 [lvl:0;]: T_CLOSE_TAG =>


    *** END LEVEL MAP ***
    *** START ADDITIONAL JS PROCESSING ***
    Process token 0: T_OPEN_TAG =>
    Process token 1: T_OPEN_SQUARE_BRACKET => [
    Process token 2: T_CLOSE_SQUARE_BRACKET => ]
    Process token 3: T_CLOSE_TAG =>
    *** END ADDITIONAL JS PROCESSING ***
    [4 tokens in 1 lines]...
    DONE in < 1 second (0 errors, 0 warnings)
    Time: 0 seconds, Memory: 2.25Mb

    ------------------------------------------------------------------------

    [2010-11-10 01:02:03] squiz

    -Assigned To:
    +Assigned To: squiz


    ------------------------------------------------------------------------

    [2010-11-10 01:01:56] squiz

    -Status: Open
    +Status: Feedback
    I can't replicate this issue. Can you please use the -vv command line
    argument to generate the tokenizer output. It should look like this:

    94:trunk gsherwood$ php scripts/phpcs --standard=Generic temp.js -vv
    Registering sniffs in Generic standard... DONE (36 sniffs registered)
    Processing temp.js
    *** START TOKENIZING ***
    Process char 0 => [ (buffer: )
    * char is token, looking ahead 8 chars *
    => Looking ahead 1 chars => []
    => Looking ahead 2 chars => []\n
    * look ahead found nothing *
    => Added token T_OPEN_SQUARE_BRACKET ([)
    Process char 1 => ] (buffer: )
    * buffer possibly contains token, looking ahead 8 chars *
    => Looking ahead 1 chars => ]\n
    * look ahead found nothing *
    => Added token T_CLOSE_SQUARE_BRACKET (])
    Process char 2 => \n (buffer: )
    *** END TOKENIZING ***
    *** START BRACKET MAP ***
    => Found square bracket opener at 1
    => Found square bracket closer at 2 for 1
    *** END BRACKET MAP ***
    *** START SCOPE MAP ***
    *** END SCOPE MAP ***
    *** START LEVEL MAP ***
    Process token 0 on line 1 [lvl:0;]: T_OPEN_TAG =>
    Process token 1 on line 1 [lvl:0;]: T_OPEN_SQUARE_BRACKET => [
    Process token 2 on line 1 [lvl:0;]: T_CLOSE_SQUARE_BRACKET => ]
    Process token 3 on line 1 [lvl:0;]: T_CLOSE_TAG =>
    *** END LEVEL MAP ***
    *** START ADDITIONAL JS PROCESSING ***
    Process token 0: T_OPEN_TAG =>
    Process token 1: T_OPEN_SQUARE_BRACKET => [
    Process token 2: T_CLOSE_SQUARE_BRACKET => ]
    Process token 3: T_CLOSE_TAG =>
    *** END ADDITIONAL JS PROCESSING ***
    [4 tokens in 1 lines]...
    DONE in < 1 second (0 errors, 0 warnings)
    Time: 0 seconds, Memory: 4.25Mb

    But I'll be looking for any differences of course.

    ------------------------------------------------------------------------

    [2010-11-09 18:52:06] oliverklee

    Description:
    ------------
    1. Create a JS file "test.js" with just the contents "[]" (without the
    quotes, without any whitespace).
    2. Check that file using any Standard, e.g. the Generic standard

    Test script:
    ---------------
    []

    Expected result:
    ----------------
    Only the time and memory get displayed.

    Actual result:
    --------------
    klee@dmok:~$ phpcs --standard=Generic /tmp/test.js
    PHP Notice: Undefined offset: 2 in
    /usr/share/php/PHP/CodeSniffer/Tokenizers/JS.php on line
    444
    Time: 0 seconds, Memory: 2.25Mb

    This is the offending line:


    if (in_array(strtolower($buffer), $tokenTypes) ===
    true
    && (preg_match('|[a-zA-z0-9_]|', $char) === 0
    preg_match('|[a-zA-z0-9_]|', $chars[($i + 1)])
    === 0)

    The $i + 1 is the offset that is too big.

    ------------------------------------------------------------------------

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppear-bugs @
categoriesphp
postedNov 10, '10 at 12:01a
activeNov 11, '10 at 11:22p
posts4
users2
websitepear.php.net

2 users in discussion

Gsherwood: 3 posts Oliver Klee: 1 post

People

Translate

site design / logo © 2022 Grokbase