FAQ
http://nodejs.org/api/stream.html#stream_event_data

In 99%, this pattern is used to glue POST data:

request.on('data', function(chunk) {
body += chunk;
});


But that can cause broken 2-bytes unicode chars. Calling setEncoding() does
not helps.
IMHO, that's not correct - if function returns string, that string should
be valid.

Is it intentional behaviour or error?

--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

Search Discussions

  • Mscdex at Oct 17, 2012 at 10:48 pm

    On Oct 17, 3:35 pm, Vitaly Puzrin wrote:
    But that can cause broken 2-bytes unicode chars. Calling setEncoding() does
    not helps.
    IMHO, that's not correct - if function returns string, that string should
    be valid.
    Why not use Buffers directly and either create a single buffer the
    size of Content-Length (if defined) or store incoming Buffers into an
    array and Buffer.concat() them together at the end? Then you can
    convert to a string afterwards if you need to.

    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Vitaly Puzrin at Oct 17, 2012 at 10:41 pm
    This question is not about buffers. It's about strings.

    If official API says, that handler can operate with strings, i expect,
    those strings are always valid.


    среда, 17 октября 2012 г., 23:49:59 UTC+4 пользователь mscdex написал:
    On Oct 17, 3:35 pm, Vitaly Puzrin wrote:
    But that can cause broken 2-bytes unicode chars. Calling setEncoding() does
    not helps.
    IMHO, that's not correct - if function returns string, that string should
    be valid.
    Why not use Buffers directly and either create a single buffer the
    size of Content-Length (if defined) or store incoming Buffers into an
    array and Buffer.concat() them together at the end? Then you can
    convert to a string afterwards if you need to.
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Matt at Oct 17, 2012 at 9:27 pm

    On Wed, Oct 17, 2012 at 4:01 PM, Vitaly Puzrin wrote:

    This question is not about buffers. It's about strings.

    If official API says, that handler can operate with strings, i expect,
    those strings are always valid.
    Then your expectations are too high.

    Internally it deals with bytes, and converts them directly to strings
    without any care for making sure the strings are valid.

    If you need valid strings you need to receive and process the entire buffer
    as one.

    Matt.

    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Vitaly Puzrin at Oct 17, 2012 at 9:04 pm
    Do you really think, that eхpectiotion to get correct strings instead of
    crap from official API is too high :) ?

    Returning strings is officially documented feature. It sould work
    correctly, or should be removed, to not confuse people.
    As i said, 99% examples in internet about joining node chunks uses strings.
    That's a reality.

    I know how to work with buffers. My point is, that API should be fixed.

    четверг, 18 октября 2012 г., 0:23:38 UTC+4 пользователь Matt Sergeant
    написал:
    On Wed, Oct 17, 2012 at 4:01 PM, Vitaly Puzrin <vit...@rcdesign.ru<javascript:>
    wrote:
    This question is not about buffers. It's about strings.

    If official API says, that handler can operate with strings, i expect,
    those strings are always valid.
    Then your expectations are too high.

    Internally it deals with bytes, and converts them directly to strings
    without any care for making sure the strings are valid.

    If you need valid strings you need to receive and process the entire
    buffer as one.

    Matt.
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Ben Noordhuis at Oct 18, 2012 at 12:05 am

    On Wed, Oct 17, 2012 at 11:04 PM, Vitaly Puzrin wrote:
    Do you really think, that eхpectiotion to get correct strings instead of
    crap from official API is too high :) ?

    Returning strings is officially documented feature. It sould work correctly,
    or should be removed, to not confuse people.
    As i said, 99% examples in internet about joining node chunks uses strings.
    That's a reality.

    I know how to work with buffers. My point is, that API should be fixed.
    I agree it's something that should be addressed or clarified in the
    documentation. Can you open an issue or send a pull request?

    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Nathan Rajlich at Oct 18, 2012 at 12:45 am
    I don't get it. This is what string_decoder was designed for. What is
    the problem?
    On Wed, Oct 17, 2012 at 2:09 PM, Ben Noordhuis wrote:
    On Wed, Oct 17, 2012 at 11:04 PM, Vitaly Puzrin wrote:
    Do you really think, that eхpectiotion to get correct strings instead of
    crap from official API is too high :) ?

    Returning strings is officially documented feature. It sould work correctly,
    or should be removed, to not confuse people.
    As i said, 99% examples in internet about joining node chunks uses strings.
    That's a reality.

    I know how to work with buffers. My point is, that API should be fixed.
    I agree it's something that should be addressed or clarified in the
    documentation. Can you open an issue or send a pull request?

    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Matt at Oct 18, 2012 at 3:08 am

    On Wed, Oct 17, 2012 at 5:04 PM, Vitaly Puzrin wrote:

    Do you really think, that eхpectiotion to get correct strings instead of
    crap from official API is too high :) ?
    Well sort of. Unicode is hard. I'm glad you get it, but most people don't,
    including the original people who created Node (no offence intended - it's
    just not something that was thought through). Even if these bugs are worked
    through, Javascript's Unicode support is so appallingly bad that you
    probably want to look elsewhere anyway.

    Returning strings is officially documented feature. It sould work
    correctly, or should be removed, to not confuse people.
    Fixing it isn't easy, and would slow down performance for anyone using
    strings in the API, which is considered bad in the Node world.

    Removing it would be a reasonable solution, but I don't see it happening.

    As bnoordhuis said - file an issue. At least it can be tracked that way. I
    just don't see it being very easy to fix.

    Matt.

    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Marco Rogers at Oct 18, 2012 at 4:19 am
    I tend to agree with the OP. If node supports strings, they should be valid
    strings. The name setEncoding implies that we plan to respect that
    encoding. Anything else is surprising. I think we're all sympathetic to the
    perf arguments. But that's something that can be worked on. I'd like to see
    this fixed. Maybe turn string_decoder on by default with an option to turn
    it off if you're a masochist?

    :Marco
    On Wednesday, October 17, 2012 12:35:47 PM UTC-7, Vitaly Puzrin wrote:

    http://nodejs.org/api/stream.html#stream_event_data

    In 99%, this pattern is used to glue POST data:

    request.on('data', function(chunk) {
    body += chunk;
    });


    But that can cause broken 2-bytes unicode chars. Calling setEncoding()does not helps.
    IMHO, that's not correct - if function returns string, that string should
    be valid.

    Is it intentional behaviour or error?
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Nathan Rajlich at Oct 18, 2012 at 5:28 am
    Seriously I'm lost. Can somebody show me some sample code to
    demonstrate the problem?

    Here's the issue on the tracker btw: https://github.com/joyent/node/issues/4160
    On Wed, Oct 17, 2012 at 9:19 PM, Marco Rogers wrote:
    I tend to agree with the OP. If node supports strings, they should be valid
    strings. The name setEncoding implies that we plan to respect that encoding.
    Anything else is surprising. I think we're all sympathetic to the perf
    arguments. But that's something that can be worked on. I'd like to see this
    fixed. Maybe turn string_decoder on by default with an option to turn it off
    if you're a masochist?

    :Marco

    On Wednesday, October 17, 2012 12:35:47 PM UTC-7, Vitaly Puzrin wrote:

    http://nodejs.org/api/stream.html#stream_event_data

    In 99%, this pattern is used to glue POST data:

    request.on('data', function(chunk) {
    body += chunk;
    });


    But that can cause broken 2-bytes unicode chars. Calling setEncoding()
    does not helps.
    IMHO, that's not correct - if function returns string, that string should
    be valid.

    Is it intentional behaviour or error?
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines:
    https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Vitaly Puzrin at Oct 18, 2012 at 5:34 am
    Yes, i can't quickly reproduce bug in my current enviroment. That's a bit
    strange.
    Need some time to ask people, who confirmed problem, and comfirmed, that
    they didn't missed setEncoding().

    четверг, 18 октября 2012 г., 9:29:12 UTC+4 пользователь Nathan Rajlich
    написал:
    Seriously I'm lost. Can somebody show me some sample code to
    demonstrate the problem?

    Here's the issue on the tracker btw:
    https://github.com/joyent/node/issues/4160
    On Wed, Oct 17, 2012 at 9:19 PM, Marco Rogers wrote:
    I tend to agree with the OP. If node supports strings, they should be valid
    strings. The name setEncoding implies that we plan to respect that encoding.
    Anything else is surprising. I think we're all sympathetic to the perf
    arguments. But that's something that can be worked on. I'd like to see this
    fixed. Maybe turn string_decoder on by default with an option to turn it off
    if you're a masochist?

    :Marco

    On Wednesday, October 17, 2012 12:35:47 PM UTC-7, Vitaly Puzrin wrote:

    http://nodejs.org/api/stream.html#stream_event_data

    In 99%, this pattern is used to glue POST data:

    request.on('data', function(chunk) {
    body += chunk;
    });


    But that can cause broken 2-bytes unicode chars. Calling setEncoding()
    does not helps.
    IMHO, that's not correct - if function returns string, that string
    should
    be valid.

    Is it intentional behaviour or error?
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines:
    https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nod...@googlegroups.com<javascript:>
    To unsubscribe from this group, send email to
    nodejs+un...@googlegroups.com <javascript:>
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Vitaly Puzrin at Oct 18, 2012 at 1:08 pm
    https://gist.github.com/3910923

    Seems to work. Sorry for noise.

    четверг, 18 октября 2012 г., 9:34:23 UTC+4 пользователь Vitaly Puzrin
    написал:
    Yes, i can't quickly reproduce bug in my current enviroment. That's a bit
    strange.
    Need some time to ask people, who confirmed problem, and comfirmed, that
    they didn't missed setEncoding().

    четверг, 18 октября 2012 г., 9:29:12 UTC+4 пользователь Nathan Rajlich
    написал:
    Seriously I'm lost. Can somebody show me some sample code to
    demonstrate the problem?

    Here's the issue on the tracker btw:
    https://github.com/joyent/node/issues/4160

    On Wed, Oct 17, 2012 at 9:19 PM, Marco Rogers <marco....@gmail.com>
    wrote:
    I tend to agree with the OP. If node supports strings, they should be valid
    strings. The name setEncoding implies that we plan to respect that encoding.
    Anything else is surprising. I think we're all sympathetic to the perf
    arguments. But that's something that can be worked on. I'd like to see this
    fixed. Maybe turn string_decoder on by default with an option to turn it off
    if you're a masochist?

    :Marco

    On Wednesday, October 17, 2012 12:35:47 PM UTC-7, Vitaly Puzrin wrote:

    http://nodejs.org/api/stream.html#stream_event_data

    In 99%, this pattern is used to glue POST data:

    request.on('data', function(chunk) {
    body += chunk;
    });


    But that can cause broken 2-bytes unicode chars. Calling setEncoding()
    does not helps.
    IMHO, that's not correct - if function returns string, that string
    should
    be valid.

    Is it intentional behaviour or error?
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines:
    https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nod...@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+un...@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Ben Noordhuis at Oct 18, 2012 at 11:59 am

    On Thu, Oct 18, 2012 at 7:28 AM, Nathan Rajlich wrote:
    Seriously I'm lost. Can somebody show me some sample code to
    demonstrate the problem?

    Here's the issue on the tracker btw: https://github.com/joyent/node/issues/4160
    The bug is in the documentation, there's a number of bad examples in there.

    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Isaac Schlueter at Oct 18, 2012 at 12:18 pm
    Here's a proof that it works as intended:

    https://gist.github.com/3911308

    On Thu, Oct 18, 2012 at 11:04 AM, Ben Noordhuis wrote:
    On Thu, Oct 18, 2012 at 7:28 AM, Nathan Rajlich wrote:
    Seriously I'm lost. Can somebody show me some sample code to
    demonstrate the problem?

    Here's the issue on the tracker btw: https://github.com/joyent/node/issues/4160
    The bug is in the documentation, there's a number of bad examples in there.

    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Marco Rogers at Oct 18, 2012 at 5:36 am
    Turns out I was full of shit here. Sorry for helping stir things up without
    doing my homework.

    https://github.com/joyent/node/issues/4160#issuecomment-9553750

    :Marco
    On Wednesday, October 17, 2012 9:19:34 PM UTC-7, Marco Rogers wrote:

    I tend to agree with the OP. If node supports strings, they should be
    valid strings. The name setEncoding implies that we plan to respect that
    encoding. Anything else is surprising. I think we're all sympathetic to the
    perf arguments. But that's something that can be worked on. I'd like to see
    this fixed. Maybe turn string_decoder on by default with an option to turn
    it off if you're a masochist?

    :Marco
    On Wednesday, October 17, 2012 12:35:47 PM UTC-7, Vitaly Puzrin wrote:

    http://nodejs.org/api/stream.html#stream_event_data

    In 99%, this pattern is used to glue POST data:

    request.on('data', function(chunk) {
    body += chunk;
    });


    But that can cause broken 2-bytes unicode chars. Calling setEncoding()does not helps.
    IMHO, that's not correct - if function returns string, that string should
    be valid.

    Is it intentional behaviour or error?
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupnodejs @
categoriesnodejs
postedOct 17, '12 at 7:35p
activeOct 18, '12 at 1:08p
posts15
users7
websitenodejs.org
irc#node.js

People

Translate

site design / logo © 2022 Grokbase