FAQ
Hi,

I have a file of integers that I want to read into a var x[]int

I cannot find a better way than reading the file contents, cast to string,
strings.split with '\n', and iterate each entry and append to a slice.

Is there a better way ?

Also, is there a way to 'prepend' to a slice instead of append ? I can use
some programming logic and allocate a array and then assign each element
inside a loop. But I want to know if there is any other way to prepend
instead of append items to a slice ?

Or, Is append not a costly operation in golang slices, that a need for
prepend is not there at all ?

Please explain and help.

Thanks.

Sankar

Search Discussions

  • Sankar P at Sep 6, 2012 at 9:18 am

    On Thu, Sep 6, 2012 at 2:21 PM, Sankar wrote:
    Hi,

    I have a file of integers that I want to read into a var x[]int

    I cannot find a better way than reading the file contents, cast to string,
    strings.split with '\n', and iterate each entry and append to a slice.

    Is there a better way ?

    Also, is there a way to 'prepend' to a slice instead of append ? I can use
    some programming logic and allocate a array and then assign each element
    inside a loop. But I want to know if there is any other way to prepend
    instead of append items to a slice ?

    Or, Is append not a costly operation in golang slices, that a need for
    prepend is not there at all ?
    I realized that since the slice is backed up by an array, if we
    allocate the slice with enough length, then the append operation will
    not be costly. If we just realloc for every append, then it will be a
    costly operation.
  • Caleb Doxsey at Sep 6, 2012 at 1:37 pm
    You may also want to use the bufio package's ReadLine method:

    http://golang.org/pkg/bufio/#Reader.ReadLine

    Then you don't have to store the entire file contents in memory. Doing this
    you won't know how many numbers there are ahead of time... but you could do
    something like add 100 integers to the array when its full instead of just
    1. (Kinda depends on what you're trying to do)
    On Thursday, September 6, 2012 3:10:21 AM UTC-6, Sankar wrote:
    On Thu, Sep 6, 2012 at 2:21 PM, Sankar wrote:
    Hi,

    I have a file of integers that I want to read into a var x[]int

    I cannot find a better way than reading the file contents, cast to string,
    strings.split with '\n', and iterate each entry and append to a slice.

    Is there a better way ?

    Also, is there a way to 'prepend' to a slice instead of append ? I can use
    some programming logic and allocate a array and then assign each element
    inside a loop. But I want to know if there is any other way to prepend
    instead of append items to a slice ?

    Or, Is append not a costly operation in golang slices, that a need for
    prepend is not there at all ?
    I realized that since the slice is backed up by an array, if we
    allocate the slice with enough length, then the append operation will
    not be costly. If we just realloc for every append, then it will be a
    costly operation.

    --
    Sankar P
    http://psankar.blogspot.com
  • Sankar P at Sep 6, 2012 at 2:05 pm
    Thanks. This is useful. But for my case, reading at one stretch will be better.
    On Thu, Sep 6, 2012 at 7:07 PM, Caleb Doxsey wrote:
    You may also want to use the bufio package's ReadLine method:

    http://golang.org/pkg/bufio/#Reader.ReadLine

    Then you don't have to store the entire file contents in memory. Doing this
    you won't know how many numbers there are ahead of time... but you could do
    something like add 100 integers to the array when its full instead of just
    1. (Kinda depends on what you're trying to do)
    On Thursday, September 6, 2012 3:10:21 AM UTC-6, Sankar wrote:
    On Thu, Sep 6, 2012 at 2:21 PM, Sankar wrote:
    Hi,

    I have a file of integers that I want to read into a var x[]int

    I cannot find a better way than reading the file contents, cast to
    string,
    strings.split with '\n', and iterate each entry and append to a slice.

    Is there a better way ?

    Also, is there a way to 'prepend' to a slice instead of append ? I can
    use
    some programming logic and allocate a array and then assign each element
    inside a loop. But I want to know if there is any other way to prepend
    instead of append items to a slice ?

    Or, Is append not a costly operation in golang slices, that a need for
    prepend is not there at all ?
    I realized that since the slice is backed up by an array, if we
    allocate the slice with enough length, then the append operation will
    not be costly. If we just realloc for every append, then it will be a
    costly operation.

    --
    Sankar P
    http://psankar.blogspot.com


    --
    Sankar P
    http://psankar.blogspot.com
  • Minux at Sep 6, 2012 at 3:28 pm

    On Thu, Sep 6, 2012 at 9:37 PM, Caleb Doxsey wrote:

    You may also want to use the bufio package's ReadLine method:

    http://golang.org/pkg/bufio/#Reader.ReadLine
    FYI, Reader.ReadLine is too primitive and its use is discouraged, should
    consider
    using Reader.ReadBytes('\n') or Reader.ReadString('\n') instead.

    This is documented in the tip (should appear in Go 1.0.3):
    http://tip.golang.org/pkg/bufio/#Reader.ReadLine
  • Sonia Keys at Sep 7, 2012 at 1:59 pm
    You have identified a good way to do this. Read with ioutil.ReadFile,
    convert to string, split with strings.Split "\n", make []int of the desired
    (or at least sufficient) size, then assign the values in a loop. I would
    use strconv.Atoi (and perhaps strings.TrimSpace first). It might sound
    like a lot of code, but you are specifying exactly what the code should do,
    and the benefits are knowing that your program is efficient, and having
    complete control over your handling of invalid or unexpected data.

    To answer questions about append though, append is an efficient way to
    build up a list, as long as you are appending to the end. Append uses a
    scheme of over-allocating by increasing amounts so that it doesn't have to
    allocate every time. Alternatively, as you noted, you can allocate it with
    sufficient capacity to begin with.

    You can use append to build up a list from the front though.

    list = append(<list type>{newElement}, list...)

    will do the job, but now you really are allocating for every element added,
    and generating lots of garbage in the process. Sometimes it's convenient,
    but usually you would do better to find a different algorithm. If nothing
    else, building the list from the end, then reversing it in place would be
    more efficient in most cases.
    On Thursday, September 6, 2012 5:10:21 AM UTC-4, Sankar wrote:
    On Thu, Sep 6, 2012 at 2:21 PM, Sankar wrote:
    Hi,

    I have a file of integers that I want to read into a var x[]int

    I cannot find a better way than reading the file contents, cast to string,
    strings.split with '\n', and iterate each entry and append to a slice.

    Is there a better way ?

    Also, is there a way to 'prepend' to a slice instead of append ? I can use
    some programming logic and allocate a array and then assign each element
    inside a loop. But I want to know if there is any other way to prepend
    instead of append items to a slice ?

    Or, Is append not a costly operation in golang slices, that a need for
    prepend is not there at all ?
    I realized that since the slice is backed up by an array, if we
    allocate the slice with enough length, then the append operation will
    not be costly. If we just realloc for every append, then it will be a
    costly operation.

    --
    Sankar P
    http://psankar.blogspot.com
  • Larry Clapp at Sep 7, 2012 at 3:51 pm

    On Friday, September 7, 2012 9:59:43 AM UTC-4, Sonia Keys wrote:
    If nothing else, building the list from the end, then reversing it in
    place would be more efficient in most cases.
    This is a common idiom in Lisp[1]: Add items to the front of your list,
    then reverse it when you return from the function. In Lisp, at least, it's
    pretty efficient, as I understand it.

    (In Go, as you say, you might instead add items to the rear of your array
    or slice, and then reverse them when you return.)

    -- Larry


    [1] Common Lisp
  • Sankar P at Sep 7, 2012 at 4:09 pm
    thanks guys. I infact did allocate max size initially and then used
    append so that there will be no realloc garbage etc.
    On Fri, Sep 7, 2012 at 9:21 PM, Larry Clapp wrote:
    On Friday, September 7, 2012 9:59:43 AM UTC-4, Sonia Keys wrote:

    If nothing else, building the list from the end, then reversing it in
    place would be more efficient in most cases.

    This is a common idiom in Lisp[1]: Add items to the front of your list, then
    reverse it when you return from the function. In Lisp, at least, it's
    pretty efficient, as I understand it.

    (In Go, as you say, you might instead add items to the rear of your array or
    slice, and then reverse them when you return.)

    -- Larry


    [1] Common Lisp


    --
    Sankar P
    http://psankar.blogspot.com

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedSep 6, '12 at 8:51a
activeSep 7, '12 at 4:09p
posts8
users5
websitegolang.org

People

Translate

site design / logo © 2022 Grokbase