FAQ
There is bufio.ReadLine(), but that will only give you a partial line
if the line is very long. And it doesn't work with Mac line-endings,
because it only checks for Unix and DOS line-endings.

The doc for bufio.ReadLine() says most people should just use
bufio.ReadString('\n') instead, but that only works with Unix line-
endings.

What I would want is a function that returns a single line, all line-
endings removed, either \n, \r, or \r\n (even \n\r ?), and complete
lines, not that stuff with prefixes. And it should return the last
line without error if it has non-zero length, and is missing a line-
ending.

Sure I can write this function myself, but I think it is a basic
enough functionality to belong in the standard library.

--

Search Discussions

  • Rory McGuire at Nov 1, 2012 at 8:51 am
    Which limits should this new ReadLine function have?

    The current built in solutions are the simplest and cleanest, and they make
    people that
    want to write their own version have to think about it.


    On Thursday, 1 November 2012 01:23:43 UTC+2, Peter Kleiweg wrote:

    There is bufio.ReadLine(), but that will only give you a partial line
    if the line is very long. And it doesn't work with Mac line-endings,
    because it only checks for Unix and DOS line-endings.

    The doc for bufio.ReadLine() says most people should just use
    bufio.ReadString('\n') instead, but that only works with Unix line-
    endings.

    What I would want is a function that returns a single line, all line-
    endings removed, either \n, \r, or \r\n (even \n\r ?), and complete
    lines, not that stuff with prefixes. And it should return the last
    line without error if it has non-zero length, and is missing a line-
    ending.

    Sure I can write this function myself, but I think it is a basic
    enough functionality to belong in the standard library.
    --
  • Andrew Gerrand at Nov 1, 2012 at 9:04 am
    ReadString and ReadBytes don't have a limit. IIUC, Peter is asking for a
    ReadLine that doesn't have a limit.

    Andrew

    On 1 November 2012 19:51, Rory McGuire wrote:

    Which limits should this new ReadLine function have?

    The current built in solutions are the simplest and cleanest, and they
    make people that
    want to write their own version have to think about it.


    On Thursday, 1 November 2012 01:23:43 UTC+2, Peter Kleiweg wrote:

    There is bufio.ReadLine(), but that will only give you a partial line
    if the line is very long. And it doesn't work with Mac line-endings,
    because it only checks for Unix and DOS line-endings.

    The doc for bufio.ReadLine() says most people should just use
    bufio.ReadString('\n') instead, but that only works with Unix line-
    endings.

    What I would want is a function that returns a single line, all line-
    endings removed, either \n, \r, or \r\n (even \n\r ?), and complete
    lines, not that stuff with prefixes. And it should return the last
    line without error if it has non-zero length, and is missing a line-
    ending.

    Sure I can write this function myself, but I think it is a basic
    enough functionality to belong in the standard library.
    --

    --
  • Rory McGuire at Nov 1, 2012 at 9:09 am
    Exactly, so anyone that can supply this fictional reader with data can
    cause the OS to run out of memory.
    On Thu, Nov 1, 2012 at 11:03 AM, Andrew Gerrand wrote:

    ReadString and ReadBytes don't have a limit. IIUC, Peter is asking for a
    ReadLine that doesn't have a limit.

    Andrew

    On 1 November 2012 19:51, Rory McGuire wrote:

    Which limits should this new ReadLine function have?

    The current built in solutions are the simplest and cleanest, and they
    make people that
    want to write their own version have to think about it.


    On Thursday, 1 November 2012 01:23:43 UTC+2, Peter Kleiweg wrote:

    There is bufio.ReadLine(), but that will only give you a partial line
    if the line is very long. And it doesn't work with Mac line-endings,
    because it only checks for Unix and DOS line-endings.

    The doc for bufio.ReadLine() says most people should just use
    bufio.ReadString('\n') instead, but that only works with Unix line-
    endings.

    What I would want is a function that returns a single line, all line-
    endings removed, either \n, \r, or \r\n (even \n\r ?), and complete
    lines, not that stuff with prefixes. And it should return the last
    line without error if it has non-zero length, and is missing a line-
    ending.

    Sure I can write this function myself, but I think it is a basic
    enough functionality to belong in the standard library.
    --

    --
  • Andrew Gerrand at Nov 1, 2012 at 9:20 am
    Well, yes. But I don't see how that's different to ReadBytes or ReadString,
    which already exist.

    It's a shame that ReadLine is named ReadLine, because ReadLine would be a
    great name for the suggested function.

    Andrew

    On 1 November 2012 20:09, Rory McGuire wrote:

    Exactly, so anyone that can supply this fictional reader with data can
    cause the OS to run out of memory.

    On Thu, Nov 1, 2012 at 11:03 AM, Andrew Gerrand wrote:

    ReadString and ReadBytes don't have a limit. IIUC, Peter is asking for a
    ReadLine that doesn't have a limit.

    Andrew

    On 1 November 2012 19:51, Rory McGuire wrote:

    Which limits should this new ReadLine function have?

    The current built in solutions are the simplest and cleanest, and they
    make people that
    want to write their own version have to think about it.


    On Thursday, 1 November 2012 01:23:43 UTC+2, Peter Kleiweg wrote:

    There is bufio.ReadLine(), but that will only give you a partial line
    if the line is very long. And it doesn't work with Mac line-endings,
    because it only checks for Unix and DOS line-endings.

    The doc for bufio.ReadLine() says most people should just use
    bufio.ReadString('\n') instead, but that only works with Unix line-
    endings.

    What I would want is a function that returns a single line, all line-
    endings removed, either \n, \r, or \r\n (even \n\r ?), and complete
    lines, not that stuff with prefixes. And it should return the last
    line without error if it has non-zero length, and is missing a line-
    ending.

    Sure I can write this function myself, but I think it is a basic
    enough functionality to belong in the standard library.
    --

    --
  • Rory McGuire at Nov 1, 2012 at 10:00 am
    :D good point, you win.

    I think there should at least be some limit, in those functions to though.

    I doubt a valid line will ever be longer than 10000 runes. I use 1.0.3 and
    there is no limit in ReadBytes or ReadSlice.

    How would one use ReadBytes safely esp. in a server that relies on it (such
    as telnet)?

    Thanks,
    On Thu, Nov 1, 2012 at 11:20 AM, Andrew Gerrand wrote:

    Well, yes. But I don't see how that's different to ReadBytes or
    ReadString, which already exist.

    It's a shame that ReadLine is named ReadLine, because ReadLine would be a
    great name for the suggested function.

    Andrew

    On 1 November 2012 20:09, Rory McGuire wrote:

    Exactly, so anyone that can supply this fictional reader with data can
    cause the OS to run out of memory.

    On Thu, Nov 1, 2012 at 11:03 AM, Andrew Gerrand wrote:

    ReadString and ReadBytes don't have a limit. IIUC, Peter is asking for a
    ReadLine that doesn't have a limit.

    Andrew

    On 1 November 2012 19:51, Rory McGuire wrote:

    Which limits should this new ReadLine function have?

    The current built in solutions are the simplest and cleanest, and they
    make people that
    want to write their own version have to think about it.


    On Thursday, 1 November 2012 01:23:43 UTC+2, Peter Kleiweg wrote:

    There is bufio.ReadLine(), but that will only give you a partial line
    if the line is very long. And it doesn't work with Mac line-endings,
    because it only checks for Unix and DOS line-endings.

    The doc for bufio.ReadLine() says most people should just use
    bufio.ReadString('\n') instead, but that only works with Unix line-
    endings.

    What I would want is a function that returns a single line, all line-
    endings removed, either \n, \r, or \r\n (even \n\r ?), and complete
    lines, not that stuff with prefixes. And it should return the last
    line without error if it has non-zero length, and is missing a line-
    ending.

    Sure I can write this function myself, but I think it is a basic
    enough functionality to belong in the standard library.
    --

    --
  • Kevin Gillette at Nov 1, 2012 at 10:05 am
    You would use it safely by constraining the input or using ReadLine.
    ReadString and friends are convenience methods, and aren't for heavy
    lifting. If you're not scanning for a line, you'd might as well use Read()
    or ReadByte() or ReadRune(), depending.
    On Thursday, November 1, 2012 3:54:52 AM UTC-6, Rory McGuire wrote:

    :D good point, you win.

    I think there should at least be some limit, in those functions to though.

    I doubt a valid line will ever be longer than 10000 runes. I use 1.0.3 and
    there is no limit in ReadBytes or ReadSlice.

    How would one use ReadBytes safely esp. in a server that relies on it
    (such as telnet)?
    --
  • Rory McGuire at Nov 1, 2012 at 10:52 am
    I'm saying that if the standard library wants to be used for servers surely
    it should by default do the right thing
    for something as simple as reading a line, even bradfitz on github used
    bufio.ReadSlice with his smtp server.
    Anyone making a server HTTP, SMTP, etc which use lines for part of the
    input would likely use the standard
    library functions, and in so doing make their servers vulnerable.

    I think the simplest solution that gets everyone thinking about it is to
    make the max line length an argument.
    No confusion then. The best solution is that ReadBytes takes a slice as an
    argument ReadBytes(make([]byte, 0, 1024), "\r\n")
      because that would have no hidden allocation.

    Apologies for hijacking the thread.

    :D Cheers,


    On Thu, Nov 1, 2012 at 12:05 PM, Kevin Gillette
    wrote:
    You would use it safely by constraining the input or using ReadLine.
    ReadString and friends are convenience methods, and aren't for heavy
    lifting. If you're not scanning for a line, you'd might as well use Read()
    or ReadByte() or ReadRune(), depending.

    On Thursday, November 1, 2012 3:54:52 AM UTC-6, Rory McGuire wrote:

    :D good point, you win.

    I think there should at least be some limit, in those functions to though.

    I doubt a valid line will ever be longer than 10000 runes. I use 1.0.3
    and there is no limit in ReadBytes or ReadSlice.

    How would one use ReadBytes safely esp. in a server that relies on it
    (such as telnet)?
    --

    <div>
    Rory McGuire
    ClearFormat - Research and Development
    UK : 44 870 224 0424
    USA : 1 877 842 6286
    RSA : 27 21 466 9400
    Email: rmcguire@clearformat.com
    Website: www.clearformat.com
    </div>
    www.clearformat.com: http://fwd.clearformat.com/8XmSh

    --
  • Andrew Gerrand at Nov 1, 2012 at 11:07 am

    On 1 November 2012 20:54, Rory McGuire wrote:

    :D good point, you win.

    I think there should at least be some limit, in those functions to though.

    I doubt a valid line will ever be longer than 10000 runes. I use 1.0.3 and
    there is no limit in ReadBytes or ReadSlice.

    How would one use ReadBytes safely esp. in a server that relies on it
    (such as telnet)?
    One possible approach is to wrap the reader with an io.LimitedReader before
    passing it to bufio.NewReader.

    http://golang.org/pkg/io/#LimitedReader

    And then you can re-set the N field after each read.

    Or just use ReadLine.

    Andrew

    --
  • Kevin Gillette at Nov 1, 2012 at 10:01 am
    ReadLine is the only one of the four (including ReadBytes, ReadSlice, and
    ReadString) that is actually purposed for reading lines, and without
    application-specific information, it's impossible to know how long a line
    is expected to be, and in some applications, the line length is unknowable.
    ReadLine retains the flexibility to be useful in the worst input case (when
    you don't know the line length and it may be very long) or the best
    algorithmic case (when you can process the line in one pass, without
    needing to store it fully).

    If an application has clear constraints on the input, such as a limited
    length line, or the need to buffer the entire line, then ReadBytes,
    ReadSlice, or ReadString might as well be used.

    --
  • Yy at Nov 1, 2012 at 11:04 am

    On 1 November 2012 10:20, Andrew Gerrand wrote:

    It's a shame that ReadLine is named ReadLine, because ReadLine would be a
    great name for the suggested function.
    >

    Actually, I'd prefer:

    func (b *Reader) ReadLineBytes() (line []byte, err error)


    --
    - yiyus || JGL .

    --
  • Peter Kleiweg at Nov 1, 2012 at 11:05 am

    On 1 nov, 10:21, Andrew Gerrand wrote:

    It's a shame that ReadLine is named ReadLine, because ReadLine would be a
    great name for the suggested function.
    ReadLine should be called ReadUnixOrDOSLine, because it doesn't work
    with Mac lines.

    --
  • Andrew Gerrand at Nov 1, 2012 at 11:10 am

    On 1 November 2012 22:05, Peter Kleiweg wrote:
    On 1 nov, 10:21, Andrew Gerrand wrote:

    It's a shame that ReadLine is named ReadLine, because ReadLine would be a
    great name for the suggested function.
    ReadLine should be called ReadUnixOrDOSLine, because it doesn't work
    with Mac lines.
    MacOS 10 is Unix. Do you mean earlier versions? Does Go run on those
    systems at all?

    --
  • Peter Kleiweg at Nov 1, 2012 at 11:30 am

    On 1 nov, 12:10, Andrew Gerrand wrote:
    On 1 November 2012 22:05, Peter Kleiweg wrote:
    On 1 nov, 10:21, Andrew Gerrand wrote:

    It's a shame that ReadLine is named ReadLine, because ReadLine would be a
    great name for the suggested function.
    ReadLine should be called ReadUnixOrDOSLine, because it doesn't work
    with Mac lines.
    MacOS 10 is Unix. Do you mean earlier versions? Does Go run on those
    systems at all?
    I am talking about file formats. I am working on Linux, with text
    files that are generated on Linux, Windows, Mac. They all have
    different line-endings. Mac line-ending is a single \r , without the
    \n.

    bufio.ReadLine() only handles \n and \r\n.

    --
  • Andrew Gerrand at Nov 1, 2012 at 11:47 am
    What Mac file formats use \r as line endings? Which programs generate them?

    I just created a file in TextEdit on my Mac OS X system and it appears to
    use \n for line breaks:

    % hexdump -C test.txt
    00000000 74 65 73 74 0a 74 65 73 74 0a 74 65 73 74 0a
    test.test.test.|
    0000000f

    Andrew

    On 1 November 2012 22:30, Peter Kleiweg wrote:
    On 1 nov, 12:10, Andrew Gerrand wrote:
    On 1 November 2012 22:05, Peter Kleiweg wrote:
    On 1 nov, 10:21, Andrew Gerrand wrote:

    It's a shame that ReadLine is named ReadLine, because ReadLine would
    be a
    great name for the suggested function.
    ReadLine should be called ReadUnixOrDOSLine, because it doesn't work
    with Mac lines.
    MacOS 10 is Unix. Do you mean earlier versions? Does Go run on those
    systems at all?
    I am talking about file formats. I am working on Linux, with text
    files that are generated on Linux, Windows, Mac. They all have
    different line-endings. Mac line-ending is a single \r , without the
    \n.

    bufio.ReadLine() only handles \n and \r\n.

    --

    --
  • Nate Finch at Nov 1, 2012 at 12:02 pm
    Pre-OSX Macs did that. That's the "Mac File Format". OSX uses the "Unix
    file format". Yes it's confusing. If you have to support it, it's an extra
    headache, because otherwise, ending a line with \n hits both windows \r\n
    and unix/linux \n, and you're just done.
    On Thursday, November 1, 2012 7:47:47 AM UTC-4, Andrew Gerrand wrote:

    What Mac file formats use \r as line endings? Which programs generate
    them?

    I just created a file in TextEdit on my Mac OS X system and it appears to
    use \n for line breaks:

    % hexdump -C test.txt
    00000000 74 65 73 74 0a 74 65 73 74 0a 74 65 73 74 0a
    test.test.test.|
    0000000f

    Andrew


    On 1 November 2012 22:30, Peter Kleiweg <pkle...@xs4all.nl <javascript:>>wrote:
    On 1 nov, 12:10, Andrew Gerrand wrote:
    On 1 November 2012 22:05, Peter Kleiweg wrote:
    On 1 nov, 10:21, Andrew Gerrand wrote:

    It's a shame that ReadLine is named ReadLine, because ReadLine
    would be a
    great name for the suggested function.
    ReadLine should be called ReadUnixOrDOSLine, because it doesn't work
    with Mac lines.
    MacOS 10 is Unix. Do you mean earlier versions? Does Go run on those
    systems at all?
    I am talking about file formats. I am working on Linux, with text
    files that are generated on Linux, Windows, Mac. They all have
    different line-endings. Mac line-ending is a single \r , without the
    \n.

    bufio.ReadLine() only handles \n and \r\n.

    --

    --
  • Thomas Kappler at Nov 1, 2012 at 12:32 pm

    On Thursday, November 1, 2012 12:54:50 PM UTC+1, Nate Finch wrote:
    Pre-OSX Macs did that. That's the "Mac File Format". OSX uses the "Unix
    file format". Yes it's confusing. If you have to support it, it's an extra
    headache, because otherwise, ending a line with \n hits both windows \r\n
    and unix/linux \n, and you're just done.
    For further processing you also need to strip the \r in case of \r\n. It's
    not rocket-science, but a bit more boilerplate code than I'd want to paste
    into every program dealing with lines.


    --
  • Matt Kane's Brain at Nov 1, 2012 at 12:49 pm

    On Thu, Nov 1, 2012 at 7:46 AM, Andrew Gerrand wrote:
    What Mac file formats use \r as line endings? Which programs generate them?
    If you export a CSV or other text file from Microsoft Excel, it has \r
    instead of \n for line endings. Unfortunate!

    --
    matt kane's brain
    http://hydrogenproject.com

    --
  • Paulo Pinto at Nov 1, 2012 at 2:08 pm
    Excel showing its Mac roots! :)
    On Nov 1, 1:49 pm, "Matt Kane's Brain" wrote:
    On Thu, Nov 1, 2012 at 7:46 AM, Andrew Gerrand wrote:
    What Mac file formats use \r as line endings? Which programs generate them?
    If you export a CSV or other text file from Microsoft Excel, it has \r
    instead of \n for line endings. Unfortunate!

    --
    matt kane's brainhttp://hydrogenproject.com
    --
  • Peter Weinberger (温博格) at Nov 1, 2012 at 4:15 pm
    OS9
    On Thu, Nov 1, 2012 at 7:46 AM, Andrew Gerrand wrote:
    What Mac file formats use \r as line endings? Which programs generate them?

    I just created a file in TextEdit on my Mac OS X system and it appears to
    use \n for line breaks:

    % hexdump -C test.txt
    00000000 74 65 73 74 0a 74 65 73 74 0a 74 65 73 74 0a
    test.test.test.|
    0000000f

    Andrew

    On 1 November 2012 22:30, Peter Kleiweg wrote:
    On 1 nov, 12:10, Andrew Gerrand wrote:
    On 1 November 2012 22:05, Peter Kleiweg wrote:
    On 1 nov, 10:21, Andrew Gerrand wrote:

    It's a shame that ReadLine is named ReadLine, because ReadLine would
    be a
    great name for the suggested function.
    ReadLine should be called ReadUnixOrDOSLine, because it doesn't work
    with Mac lines.
    MacOS 10 is Unix. Do you mean earlier versions? Does Go run on those
    systems at all?
    I am talking about file formats. I am working on Linux, with text
    files that are generated on Linux, Windows, Mac. They all have
    different line-endings. Mac line-ending is a single \r , without the
    \n.

    bufio.ReadLine() only handles \n and \r\n.

    --
    --
    --
  • Andrew Gerrand at Nov 1, 2012 at 11:52 am
    I know that OS9 uses \r, but I am wondering what file formats Peter K is
    interested in reading.

    I think it's okay that Go's ReadLine doesn't support file formats generated
    by an operating system that has been obsolete for 10 years.

    On 1 November 2012 22:48, Peter Weinberger (温博格) wrote:

    OS9
    On Thu, Nov 1, 2012 at 7:46 AM, Andrew Gerrand wrote:
    What Mac file formats use \r as line endings? Which programs generate them?
    I just created a file in TextEdit on my Mac OS X system and it appears to
    use \n for line breaks:

    % hexdump -C test.txt
    00000000 74 65 73 74 0a 74 65 73 74 0a 74 65 73 74 0a
    test.test.test.|
    0000000f

    Andrew

    On 1 November 2012 22:30, Peter Kleiweg wrote:
    On 1 nov, 12:10, Andrew Gerrand wrote:
    On 1 November 2012 22:05, Peter Kleiweg wrote:
    On 1 nov, 10:21, Andrew Gerrand wrote:

    It's a shame that ReadLine is named ReadLine, because ReadLine
    would
    be a
    great name for the suggested function.
    ReadLine should be called ReadUnixOrDOSLine, because it doesn't work
    with Mac lines.
    MacOS 10 is Unix. Do you mean earlier versions? Does Go run on those
    systems at all?
    I am talking about file formats. I am working on Linux, with text
    files that are generated on Linux, Windows, Mac. They all have
    different line-endings. Mac line-ending is a single \r , without the
    \n.

    bufio.ReadLine() only handles \n and \r\n.

    --
    --
    --
  • Patrick Mylund Nielsen at Nov 1, 2012 at 12:57 pm
    I don't use a Mac, so I can't say exactly, but I still deal with this
    problem regularly. IIRC the default text editor(s) (TextEdit?) let you save
    "Mac" or "UNIX" text, or with "Mac" or "UNIX" linebreaks, and the linebreak
    character becomes \r. The frequency at which I face this problem would
    suggest that it's the default in at least some of those editors, and
    Office--it's 50/50 when I'm dealing with user data from Macs. I'll try to
    find out exactly which programs the next time I see it.

    So far my fix has been something quick and ugly like perl -p -i -e
    "s/\r/\n/g" foo.csv

    On Thu, Nov 1, 2012 at 12:51 PM, Andrew Gerrand wrote:

    I know that OS9 uses \r, but I am wondering what file formats Peter K is
    interested in reading.

    I think it's okay that Go's ReadLine doesn't support file formats
    generated by an operating system that has been obsolete for 10 years.

    On 1 November 2012 22:48, Peter Weinberger (温博格) wrote:

    OS9

    On Thu, Nov 1, 2012 at 7:46 AM, Andrew Gerrand wrote:
    What Mac file formats use \r as line endings? Which programs generate them?
    I just created a file in TextEdit on my Mac OS X system and it appears to
    use \n for line breaks:

    % hexdump -C test.txt
    00000000 74 65 73 74 0a 74 65 73 74 0a 74 65 73 74 0a
    test.test.test.|
    0000000f

    Andrew

    On 1 November 2012 22:30, Peter Kleiweg wrote:
    On 1 nov, 12:10, Andrew Gerrand wrote:
    On 1 November 2012 22:05, Peter Kleiweg wrote:
    On 1 nov, 10:21, Andrew Gerrand wrote:

    It's a shame that ReadLine is named ReadLine, because ReadLine
    would
    be a
    great name for the suggested function.
    ReadLine should be called ReadUnixOrDOSLine, because it doesn't
    work
    with Mac lines.
    MacOS 10 is Unix. Do you mean earlier versions? Does Go run on those
    systems at all?
    I am talking about file formats. I am working on Linux, with text
    files that are generated on Linux, Windows, Mac. They all have
    different line-endings. Mac line-ending is a single \r , without the
    \n.

    bufio.ReadLine() only handles \n and \r\n.

    --
    --
    --

    --
  • Erwin at Nov 1, 2012 at 7:01 pm
    For some fun/misery with EOL, try reading the lines of a couple of .pdf
    files. Chances are you'll encounter some that use \n, \r, and \r\n in the
    same file!


    What Mac file formats use \r as line endings? Which programs generate them?
    I just created a file in TextEdit on my Mac OS X system and it appears to
    use \n for line breaks:

    % hexdump -C test.txt
    00000000 74 65 73 74 0a 74 65 73 74 0a 74 65 73 74 0a
    test.test.test.|
    0000000f

    --
  • Peter Kleiweg at Nov 1, 2012 at 7:36 pm
    The Go compiler can't handle source code with Mac line-endings:

         go build test.go
         can't load package: package :
         test.go:1:15: expected ';', found 'import'

    Python can.

    ****

    Here is some info on different types of line-endings:

         http://en.wikipedia.org/wiki/Newline#

    --
  • Dustin at Nov 1, 2012 at 7:40 pm

    On Thursday, November 1, 2012 12:36:48 PM UTC-7, Peter Kleiweg wrote:
    The Go compiler can't handle source code with Mac line-endings:

    go build test.go
    can't load package: package :
    test.go:1:15: expected ';', found 'import'

    Python can.
       Software that produced mac line endings had no need to exist after the
    creation of go. Considering the number of years it's been since I've seen
    a file with a mac line ending on my mac, I can't imagine it'd be a priority
    for the compiler to support such a thing.

    --
  • Dan Cross at Nov 1, 2012 at 7:40 pm

    On Thu, Nov 1, 2012 at 3:36 PM, Peter Kleiweg wrote:
    The Go compiler can't handle source code with Mac line-endings:

    go build test.go
    can't load package: package :
    test.go:1:15: expected ';', found 'import'

    Python can.

    ****

    Here is some info on different types of line-endings:

    http://en.wikipedia.org/wiki/Newline#
    This is a little silly.

    What application are you using on a platform that supports Go to write
    Go programs that saves text using MacOs 9-style line endings?

             - Dan C.

    --
  • Peter Kleiweg at Nov 1, 2012 at 7:51 pm

    On 1 nov, 20:40, Dan Cross wrote:
    On Thu, Nov 1, 2012 at 3:36 PM, Peter Kleiweg wrote:
    The Go compiler can't handle source code with Mac line-endings:
    go build test.go
    can't load package: package :
    test.go:1:15: expected ';', found 'import'
    Python can.
    ****
    Here is some info on different types of line-endings:
    http://en.wikipedia.org/wiki/Newline#
    This is a little silly.

    What application are you using on a platform that supports Go to write
    Go programs that saves text using MacOs 9-style line endings?
    I know plenty of Unix people who still use 'ed'. I don't know what Mac
    people use. I don't know about Mac people.

    --
  • Dan Cross at Nov 1, 2012 at 8:00 pm

    On Thu, Nov 1, 2012 at 3:51 PM, Peter Kleiweg wrote:
    On 1 nov, 20:40, Dan Cross wrote:
    On Thu, Nov 1, 2012 at 3:36 PM, Peter Kleiweg wrote:
    http://en.wikipedia.org/wiki/Newline#
    This is a little silly.

    What application are you using on a platform that supports Go to write
    Go programs that saves text using MacOs 9-style line endings?
    I know plenty of Unix people who still use 'ed'. I don't know what Mac
    people use.
    That's irrelevant.
    I don't know about Mac people.
    Current Mac software produces text files using the Unix line-ending convention.

    Mac OS 9 produced what you are calling "mac files," but that was
    deprecated ten years ago (read: development of Mac OS 9 stopped in
    2002). The Go suite does not support Mac OS 9, to my knowledge, so
    whether the Go compiler can read files that use the Mac OS <X line
    ending convention doesn't seem relevant to me at all. That Python can
    is also irrelevant: it's an older language that did, I believe, have a
    port to Mac OS prior to OS X.

             - Dan C.

    --
  • Job van der Zwan at Nov 1, 2012 at 8:56 pm

    On Thursday, 1 November 2012 21:00:38 UTC+1, Dan Cross wrote:
    On Thu, Nov 1, 2012 at 3:51 PM, Peter Kleiweg wrote:
    On 1 nov, 20:40, Dan Cross wrote:
    On Thu, Nov 1, 2012 at 3:36 PM, Peter Kleiweg wrote:
    http://en.wikipedia.org/wiki/Newline#
    This is a little silly.

    What application are you using on a platform that supports Go to write
    Go programs that saves text using MacOs 9-style line endings?
    I know plenty of Unix people who still use 'ed'. I don't know what Mac
    people use.
    That's irrelevant.
    I don't know about Mac people.
    Current Mac software produces text files using the Unix line-ending
    convention.

    Mac OS 9 produced what you are calling "mac files," but that was
    deprecated ten years ago (read: development of Mac OS 9 stopped in
    2002). The Go suite does not support Mac OS 9, to my knowledge, so
    whether the Go compiler can read files that use the Mac OS <X line
    ending convention doesn't seem relevant to me at all. That Python can
    is also irrelevant: it's an older language that did, I believe, have a
    port to Mac OS prior to OS X.

    - Dan C.
    I'm sure it's a legacy thing most people never have to deal with in a
    serious way, but considering Peter Kleiweg works at the department of Humanties
    Computing in Groningen <http://www.rug.nl/let/informatiekunde>, I'd expect
    very old textfiles would be quite a common thing for him to work with.

    --
  • Dan Cross at Nov 1, 2012 at 11:19 pm

    On Thu, Nov 1, 2012 at 4:56 PM, Job van der Zwan wrote:
    On Thursday, 1 November 2012 21:00:38 UTC+1, Dan Cross wrote:
    Current Mac software produces text files using the Unix line-ending
    convention.

    Mac OS 9 produced what you are calling "mac files," but that was
    deprecated ten years ago (read: development of Mac OS 9 stopped in
    2002). The Go suite does not support Mac OS 9, to my knowledge, so
    whether the Go compiler can read files that use the Mac OS <X line
    ending convention doesn't seem relevant to me at all. That Python can
    is also irrelevant: it's an older language that did, I believe, have a
    port to Mac OS prior to OS X.
    I'm sure it's a legacy thing most people never have to deal with in a
    serious way, but considering Peter Kleiweg works at the department of
    Humanties Computing in Groningen, I'd expect very old textfiles would be
    quite a common thing for him to work with.
    I'm sure he has a need for it; I'm not doubting his veracity. But an
    individual having a need for such a thing does not imply that it
    should be added to the standard library. Similarly, the standard
    library not supporting a convention not habitually used by any current
    systems not does imply that it is somehow broken or lacking proper
    functionality.

    Particularly when simple workarounds exist: chaining readers, tr
    '\015' '\012', etc.

             - Dan C.

    --
  • Kevin Gillette at Nov 2, 2012 at 2:06 pm

    On Thursday, November 1, 2012 1:36:48 PM UTC-6, Peter Kleiweg wrote:

    The Go compiler can't handle source code with Mac line-endings
    Python can.
    Python was around well before Mac OS X existed, and Python certainly did
    (if not still does) support OS 9. Also, python on OS 9 supports unix line
    endings for source files.

    I think it's rather ridiculous that the entire ecosystem hasn't reduced to
    using \n for source code files. Any _decent_ windows editor (even if it's
    merely notepad without the arbitrary patheticisms) supports unix line
    endings. Apple made it's choice. Windows of course has a legacy burden, but
    sane "text" files should not be using \n for some non-eol purpose when the
    line endings are \r\n or \r anyway, so MS could certainly change their "rt"
    file mode handling to consider \r\n and \n both line endings, and phase out
    \r\n in "wt" file modes a few years later.

    --
  • Andrew Gerrand at Nov 1, 2012 at 12:58 pm
    You could always write a converting reader that swaps lone \r bytes with
    \n, and then continue to use ReadLine:

    http://play.golang.org/p/-aYmMfYKnZ

    On 1 November 2012 22:30, Peter Kleiweg wrote:
    On 1 nov, 12:10, Andrew Gerrand wrote:
    On 1 November 2012 22:05, Peter Kleiweg wrote:
    On 1 nov, 10:21, Andrew Gerrand wrote:

    It's a shame that ReadLine is named ReadLine, because ReadLine would
    be a
    great name for the suggested function.
    ReadLine should be called ReadUnixOrDOSLine, because it doesn't work
    with Mac lines.
    MacOS 10 is Unix. Do you mean earlier versions? Does Go run on those
    systems at all?
    I am talking about file formats. I am working on Linux, with text
    files that are generated on Linux, Windows, Mac. They all have
    different line-endings. Mac line-ending is a single \r , without the
    \n.

    bufio.ReadLine() only handles \n and \r\n.

    --

    --
  • Patrick Mylund Nielsen at Nov 1, 2012 at 1:01 pm
    or throw your arms in the air and refuse to process anything but lines with
    \n in your programs. It's 2013 soon -- it's ridiculous and sad that we're
    still dealing with this problem.

    On Thu, Nov 1, 2012 at 1:57 PM, Andrew Gerrand wrote:

    You could always write a converting reader that swaps lone \r bytes with
    \n, and then continue to use ReadLine:

    http://play.golang.org/p/-aYmMfYKnZ

    On 1 November 2012 22:30, Peter Kleiweg wrote:
    On 1 nov, 12:10, Andrew Gerrand wrote:
    On 1 November 2012 22:05, Peter Kleiweg wrote:
    On 1 nov, 10:21, Andrew Gerrand wrote:

    It's a shame that ReadLine is named ReadLine, because ReadLine
    would be a
    great name for the suggested function.
    ReadLine should be called ReadUnixOrDOSLine, because it doesn't work
    with Mac lines.
    MacOS 10 is Unix. Do you mean earlier versions? Does Go run on those
    systems at all?
    I am talking about file formats. I am working on Linux, with text
    files that are generated on Linux, Windows, Mac. They all have
    different line-endings. Mac line-ending is a single \r , without the
    \n.

    bufio.ReadLine() only handles \n and \r\n.

    --

    --

    --
  • Rory McGuire at Nov 1, 2012 at 4:15 pm
    +1
    On Thu, Nov 1, 2012 at 3:00 PM, Patrick Mylund Nielsen wrote:

    or throw your arms in the air and refuse to process anything but lines
    with \n in your programs. It's 2013 soon -- it's ridiculous and sad that
    we're still dealing with this problem.

    On Thu, Nov 1, 2012 at 1:57 PM, Andrew Gerrand wrote:

    You could always write a converting reader that swaps lone \r bytes with
    \n, and then continue to use ReadLine:

    http://play.golang.org/p/-aYmMfYKnZ

    On 1 November 2012 22:30, Peter Kleiweg wrote:
    On 1 nov, 12:10, Andrew Gerrand wrote:
    On 1 November 2012 22:05, Peter Kleiweg wrote:
    On 1 nov, 10:21, Andrew Gerrand wrote:

    It's a shame that ReadLine is named ReadLine, because ReadLine
    would be a
    great name for the suggested function.
    ReadLine should be called ReadUnixOrDOSLine, because it doesn't work
    with Mac lines.
    MacOS 10 is Unix. Do you mean earlier versions? Does Go run on those
    systems at all?
    I am talking about file formats. I am working on Linux, with text
    files that are generated on Linux, Windows, Mac. They all have
    different line-endings. Mac line-ending is a single \r , without the
    \n.

    bufio.ReadLine() only handles \n and \r\n.

    --

    --

    --

    <div>
    Rory McGuire
    ClearFormat - Research and Development
    UK : 44 870 224 0424
    USA : 1 877 842 6286
    RSA : 27 21 466 9400
    Email: rmcguire@clearformat.com
    Website: www.clearformat.com
    </div>
    www.clearformat.com: http://fwd.clearformat.com/8XNn7

    --
  • Rob Pike at Nov 1, 2012 at 4:47 pm
    The right answer here requires more thought. For instance, it's
    possible that a better design is a more general approach with a
    specialization to deal with lines. The problem space is large and so
    is the design space. The proposal here addresses the immediate problem
    but not the full one.

    I've been thinking about this area for a while and it's harder than it
    looks to get right.

    -rob

    --
  • Peter Kleiweg at Nov 1, 2012 at 5:00 pm

    On 1 nov, 17:40, Rob Pike wrote:
    The right answer here requires more thought. For instance, it's
    possible that a better design is a more general approach with a
    specialization to deal with lines. The problem space is large and so
    is the design space. The proposal here addresses the immediate problem
    but not the full one.

    I've been thinking about this area for a while and it's harder than it
    looks to get right.
    Can you tell more about the full problem?

    The thing I can think of is handling text encoded in UTF-16. My
    implementation ignores that.

    --
  • Larry Clapp at Nov 1, 2012 at 1:17 pm
    Just as an aside, this is an awesome feature of Go that you can chain
    Readers this way. The more I use Go, the more I see how it embraces (what
    I think of as) the Unix philosophy: write small things and chain them
    together (e.g. pipes and such in the shell). You can do this via Readers &
    Writers, or channels, or probably lots of other things.

    I guess this is really the CSP philosophy, not so much Unix, as such.

    Regardless, it's cool. Kudos to the stdlib authors for enabling it.

    -- Larry

    On Thursday, November 1, 2012 8:58:25 AM UTC-4, Andrew Gerrand wrote:

    You could always write a converting reader that swaps lone \r bytes with
    \n, and then continue to use ReadLine:

    http://play.golang.org/p/-aYmMfYKnZ


    On 1 November 2012 22:30, Peter Kleiweg <pkle...@xs4all.nl <javascript:>>wrote:
    On 1 nov, 12:10, Andrew Gerrand wrote:
    On 1 November 2012 22:05, Peter Kleiweg wrote:
    On 1 nov, 10:21, Andrew Gerrand wrote:

    It's a shame that ReadLine is named ReadLine, because ReadLine
    would be a
    great name for the suggested function.
    ReadLine should be called ReadUnixOrDOSLine, because it doesn't work
    with Mac lines.
    MacOS 10 is Unix. Do you mean earlier versions? Does Go run on those
    systems at all?
    I am talking about file formats. I am working on Linux, with text
    files that are generated on Linux, Windows, Mac. They all have
    different line-endings. Mac line-ending is a single \r , without the
    \n.

    bufio.ReadLine() only handles \n and \r\n.

    --

    --
  • Ryan Tarpine at Nov 1, 2012 at 1:47 pm
    This is an example of the decorator pattern (
    http://en.wikipedia.org/wiki/Decorator_pattern) and this exact feature has
    been present in Java since 1.0 back in '96 :-)

    -Ryan
    On Thursday, November 1, 2012 9:17:48 AM UTC-4, Larry Clapp wrote:

    Just as an aside, this is an awesome feature of Go that you can chain
    Readers this way. The more I use Go, the more I see how it embraces (what
    I think of as) the Unix philosophy: write small things and chain them
    together (e.g. pipes and such in the shell). You can do this via Readers &
    Writers, or channels, or probably lots of other things.

    I guess this is really the CSP philosophy, not so much Unix, as such.

    Regardless, it's cool. Kudos to the stdlib authors for enabling it.

    -- Larry

    On Thursday, November 1, 2012 8:58:25 AM UTC-4, Andrew Gerrand wrote:

    You could always write a converting reader that swaps lone \r bytes with
    \n, and then continue to use ReadLine:

    http://play.golang.org/p/-aYmMfYKnZ

    On 1 November 2012 22:30, Peter Kleiweg wrote:
    On 1 nov, 12:10, Andrew Gerrand wrote:
    On 1 November 2012 22:05, Peter Kleiweg wrote:
    On 1 nov, 10:21, Andrew Gerrand wrote:

    It's a shame that ReadLine is named ReadLine, because ReadLine
    would be a
    great name for the suggested function.
    ReadLine should be called ReadUnixOrDOSLine, because it doesn't work
    with Mac lines.
    MacOS 10 is Unix. Do you mean earlier versions? Does Go run on those
    systems at all?
    I am talking about file formats. I am working on Linux, with text
    files that are generated on Linux, Windows, Mac. They all have
    different line-endings. Mac line-ending is a single \r , without the
    \n.

    bufio.ReadLine() only handles \n and \r\n.

    --

    --
  • Larry Clapp at Nov 1, 2012 at 2:22 pm
    Good point. It's nifty there, too. :)
    On Thursday, November 1, 2012 9:47:51 AM UTC-4, Ryan Tarpine wrote:

    This is an example of the decorator pattern (
    http://en.wikipedia.org/wiki/Decorator_pattern) and this exact feature
    has been present in Java since 1.0 back in '96 :-)

    -Ryan
    On Thursday, November 1, 2012 9:17:48 AM UTC-4, Larry Clapp wrote:

    Just as an aside, this is an awesome feature of Go that you can chain
    Readers this way. The more I use Go, the more I see how it embraces (what
    I think of as) the Unix philosophy: write small things and chain them
    together (e.g. pipes and such in the shell). You can do this via Readers &
    Writers, or channels, or probably lots of other things.

    I guess this is really the CSP philosophy, not so much Unix, as such.

    Regardless, it's cool. Kudos to the stdlib authors for enabling it.

    -- Larry

    On Thursday, November 1, 2012 8:58:25 AM UTC-4, Andrew Gerrand wrote:

    You could always write a converting reader that swaps lone \r bytes with
    \n, and then continue to use ReadLine:

    http://play.golang.org/p/-aYmMfYKnZ

    On 1 November 2012 22:30, Peter Kleiweg wrote:
    On 1 nov, 12:10, Andrew Gerrand wrote:
    On 1 November 2012 22:05, Peter Kleiweg wrote:
    On 1 nov, 10:21, Andrew Gerrand wrote:

    It's a shame that ReadLine is named ReadLine, because ReadLine
    would be a
    great name for the suggested function.
    ReadLine should be called ReadUnixOrDOSLine, because it doesn't work
    with Mac lines.
    MacOS 10 is Unix. Do you mean earlier versions? Does Go run on those
    systems at all?
    I am talking about file formats. I am working on Linux, with text
    files that are generated on Linux, Windows, Mac. They all have
    different line-endings. Mac line-ending is a single \r , without the
    \n.

    bufio.ReadLine() only handles \n and \r\n.

    --

    --
  • Peter Kleiweg at Nov 1, 2012 at 2:51 pm

    On 1 nov, 13:58, Andrew Gerrand wrote:
    You could always write a converting reader that swaps lone \r bytes with
    \n, and then continue to use ReadLine:

    http://play.golang.org/p/-aYmMfYKnZ
    That is not correct. When r.r.Read(b) gives an error, you should still
    process the bytes from that call.

    --
  • Dan Cross at Nov 1, 2012 at 7:39 pm

    On Thu, Nov 1, 2012 at 7:05 AM, Peter Kleiweg wrote:
    On 1 nov, 10:21, Andrew Gerrand wrote:
    It's a shame that ReadLine is named ReadLine, because ReadLine would be a
    great name for the suggested function.
    ReadLine should be called ReadUnixOrDOSLine, because it doesn't work
    with Mac lines.
    To my knowledge it won't work with EBCDIC, Baudot or the 6-bit
    character set of the UNISYS 1100 series machines without re-encoding,
    either.

    It strikes me that it is reasonable for the Go standard library to not
    support something that is a decade out of date. Go makes it fairly
    easy to solve this problem with a wrapper, and that seems sufficient
    for applications seeking to process obsolete file formats.

             - Dan C.

    --
  • Peter Kleiweg at Nov 1, 2012 at 7:49 pm

    On 1 nov, 20:39, Dan Cross wrote:
    On Thu, Nov 1, 2012 at 7:05 AM, Peter Kleiweg wrote:

    ReadLine should be called ReadUnixOrDOSLine, because it doesn't work
    with Mac lines.
    To my knowledge it won't work with EBCDIC, Baudot or the 6-bit
    character set of the UNISYS 1100 series machines without re-encoding,
    either.
    I've never come across anything like those. Mac files: regularly.

    --
  • Dan Cross at Nov 1, 2012 at 8:00 pm

    On Thu, Nov 1, 2012 at 3:49 PM, Peter Kleiweg wrote:
    On 1 nov, 20:39, Dan Cross wrote:
    On Thu, Nov 1, 2012 at 7:05 AM, Peter Kleiweg wrote:
    ReadLine should be called ReadUnixOrDOSLine, because it doesn't work
    with Mac lines.
    To my knowledge it won't work with EBCDIC, Baudot or the 6-bit
    character set of the UNISYS 1100 series machines without re-encoding,
    either.
    I've never come across anything like those. Mac files: regularly.
    Really? In what context? I don't think I've seen any since Mac OS 9
    stopped being a going concern. Where I did, I trivially converted
    them to the Unix convention.

    Put another way: what percentage of text files likely to be read by
    something that calls ReadLine use the Mac OS 9 convention? Is it
    really worth it to support an obsolete line convention? What other
    obsolete data formats should the Go standard library support? This
    doesn't make a strong case for inclusion in a standard library.

    The fact is that these files are either legacy data or produced by
    legacy tools; as such, they are more than likely a tiny percentage of
    data likely to processed by tools written in Go. More reasonable
    alternatives exist: convert to a supported line convention, or put a
    wrapper in your code.

             - Dan C.

    --
  • Jens Alfke at Nov 1, 2012 at 11:58 pm
    I agree that it would be silly for Go to add special support for the
    highly-obsolete Classic-Mac-OS line break. (Even calling it "Mac line
    breaks" or "Mac format" is misleading. This format was pretty well stamped
    out years ago. Maybe people dealing with very old files, or very old Mac
    apps, still see these sometimes, but that's a special case and it's not
    hard to preprocess the file.)

    There are some very real problems trying to support all 3 flavors. I used
    to work in the Java group at Apple in the late '90s, and saw several
    developers run into a nasty deadlock problem reading lines from network
    streams. The Java ReadLine method would support all styles of line breaks.
    This meant that when it received a CR it would read the next byte to check
    whether it was a LF, and if so skip it. Unfortunately if it was reading
    from a socket and the last thing the peer sent was a CR, because the peer
    was sending Old-Mac-format lines, then it would block waiting for another
    byte; whereas the peer had finished sending its line and was waiting for a
    response. The result was deadlock. (This was fixed in the JDK by being
    smarter about the LF, not trying to read it immediately. I'm just pointing
    this out as a pitfall one can run into trying to parse line breaks.)

    --Jens

    --
  • Peter Kleiweg at Nov 2, 2012 at 10:38 am

    On 2 nov, 00:58, Jens Alfke wrote:
    I agree that it would be silly for Go to add special support for the
    highly-obsolete Classic-Mac-OS line break.
    However highly-obsolete, it is still around. Reading files means
    reading something that already exists. It may be a document created
    moments ago, or years ago. It may be a document created moments ago by
    a program that pastes in document fragments created years ago. (I see
    this in PostScript documents: parts in the same file with different
    line-endings.) In many real applications, you need to be able to deal
    with the past.

    You need general support, not special support.

    --
  • Jens Alfke at Nov 2, 2012 at 4:02 pm

    On Nov 2, 2012, at 3:38 AM, Peter Kleiweg wrote:

    However highly-obsolete, it is still around. Reading files means
    reading something that already exists.
    To use an analogy: There are thousands of old graphics file formats floating around too, and it’s not hard to find images saved in them, but that doesn’t mean that every platform’s built-in image class/library knows how to read them all. Generally you just expect support for the ones in wide use like JPEG/PNG/GIF. If you’re writing code that needs to be able to open Targa or PICT or Xerox Alto bitmaps, you go find an external library that supports them and link with that.

    The same is true IMHO of reading text files. The bog-standard built-in ReadLine function should be expected to handle formats in current use, i.e. Unix and Windows line breaks, and 8-bit and UTF-16 encodings. Anything more exotic, like Mac-classic line breaks or Shift-JIS encoding, can be left to 3rd party packages.

    If you still really, really disagree, then I think it would be more productive for you to write an industrial-strength new ReadLine function (plus lots of unit tests, including ones for the deadlock I wrote about!) and submit a patch to the Go dev team.

    —Jens

    --
  • Agl at Nov 2, 2012 at 6:44 pm
    Crumbs, this is developing into quite a thread :)

    As the fool responsible for ReadLine, I'd like to explain why I wrote it
    the way that I did.

    I didn't support Mac line endings because, honestly, I've not seen anything
    use them in a very long time. I have a Mac and I still haven't seen them
    used.

    It only supports limited line lengths because I don't want programs that
    subtly explode and thrash the page file when fed invalid input. I figured
    that it was fairly easy to build an allocating version of ReadLine from the
    limited one, and the isPrefix flag also allows code that doesn't want to
    handle huge lines to resync. (In hindsight, perhaps I should have left the
    name ReadLine for the allocating version.) But it's tough to build the
    robust version from the allocating version so I wrote the primitive first.

    I've used ReadLine in a bunch of programs now and it's always worked just
    fine. I don't think it was a terrible design, even if it's not perfect.

    But if people want to handle UTF-16 line breaks, Mac line breaks, etc I
    suspect that's a little much for bufio. I'd suggest that might be a
    separate package, building on bufio.


    Cheers

    AGL

    --
  • Peter Kleiweg at Nov 2, 2012 at 7:34 pm

    On 2 nov, 19:44, agl wrote:

    But if people want to handle UTF-16 line breaks, Mac line breaks, etc I
    suspect that's a little much for bufio. I'd suggest that might be a
    separate package, building on bufio.
    There are some functions in bufio that go part of the way.
    ReadBytes(), ReadString(). They are limited in allowing only a single
    byte to be used as a delimiter. A more general function would allow
    sets of bytes or strings, for instance ["a", "b", "cd"]. Or even a
    regular expression as a delimiter.

    Or perhaps a more generalized stream library. What about bytes.Buffer,
    can you keep writing to it indefinately, if you read from it also?

    --
  • Peter Kleiweg at Nov 2, 2012 at 7:01 pm

    On 2 nov, 17:02, Jens Alfke wrote:

    To use an analogy: There are thousands of old graphics file formats floating around too, and it’s not hard to find images saved in them, but that doesn’t mean that every platform’s built-in image class/library knows how to read them all. Generally you just expect support for the ones in wide use like JPEG/PNG/GIF. If you’re writing code that needs to be able to open Targa or PICT or Xerox Alto bitmaps, you go find an external library that supports them and link with that.

    The same is true IMHO of reading text files. The bog-standard built-in ReadLine function should be expected to handle formats in current use, i.e. Unix and Windows line breaks, and 8-bit and UTF-16 encodings. Anything more exotic, like Mac-classic line breaks or Shift-JIS encoding, can be left to 3rd party packages.
    What is classic is a matter of perspective. I can say, we don't need
    to bother with inches and feet and yards and miles, we abandoned those
    in the dark ages, everything should be metric by now. Well, it isn't.

    An importent point is that line-end codings is not something that
    separates one document from another like EBCDIC from ASCII from Shift-
    JIS. The latter are very different beasts. But Unix, DOS or Mac ASCII
    text files, they can and do live in the same environment, usually
    without the user noticing the difference.

    But this is just part of the issue of why I started this thread.

    I have come to expect from modern programming languages to have line
    processing build in, or at least part of the standard library. See
    Perl, Python. Go only offers a partial solution. It's like that age
    old C functions, fgets(), you can get by by using a buffer that you
    think is big enough. Go is a modern languages where you don't have to
    bother about those C things like allocating and freeing memory, where
    you can just concatenate two strings together without having to worry
    about available buffer size. Why then is there no function that simply
    gives me a single line of text when I want to?

    There are other things I am missing in the standard library. Like
    handling different text encodings. Yes, EBCDIC and Shift-JIS, all
    those strange beasts, to me, that's standard toolkit stuff. But here
    we are at a point where there is no clear answer to what should be in
    the standard library. It's a matter of perspective, from a certain
    point the choices become arbitrary. Why are there all those crypto
    packages, but no OAuth functionality? Why no library for building a
    GUI (even something primitive, like tK). Why yacc, but not (f)lex?

    --
  • Agl at Nov 2, 2012 at 7:17 pm

    On Friday, November 2, 2012 3:01:53 PM UTC-4, Peter Kleiweg wrote:

    An importent point is that line-end codings is not something that
    separates one document from another like EBCDIC from ASCII from Shift-
    JIS. The latter are very different beasts. But Unix, DOS or Mac ASCII
    text files, they can and do live in the same environment, usually
    without the user noticing the difference.
    My reply was not to suggest that we shouldn't handle Mac line endings, just
    to explain why we don't. Had I known that Mac line endings weren't dead, I
    would have supported them. (And sighed, but there you go.)

    It's not clear to me whether adding support for Mac line endings in the
    stable releases is too great a semantic change.

    But since people seem to have numerous desiderata concerning line endings,
    I suspect it would be a chat with Rob and a new package.

    As for "why do we have $x, not $y?". Because we needed $x, and not yet $y.
    I'm sure we write different sorts of programs and the contents of the Go
    standard library reflect the people who wrote it and their backgrounds. I
    think that's inevitable and it doesn't suggest that $x is universally
    important, but $y isn't. We where have a hole, the subrepos can fill in.


    Cheers

    AGL

    --
  • Peter Kleiweg at Nov 2, 2012 at 10:16 pm

    On 2 nov, 20:17, agl wrote:

    As for "why do we have $x, not $y?". Because we needed $x, and not yet $y.
    I'm sure we write different sorts of programs and the contents of the Go
    standard library reflect the people who wrote it and their backgrounds. I
    think that's inevitable and it doesn't suggest that $x is universally
    important, but $y isn't. We where have a hole, the subrepos can fill in.
    I completely understand. There is no such thing as the perfect
    standard library.

    --

Related Discussions

People

Translate

site design / logo © 2021 Grokbase