FAQ
Hi

I think I must be misunderstanding something.

I have 2 buffers to double buffer data.
They are created with zero length and capacity to hold the data that will
be written to them.
buf_1 = make ([]float64, 0, n)
buf_2 = make ([]float64, 0, n)

I then select which buffer to use by assigning it to a local variable.
var curr_buffer []float64 = buf_1
var next_buffer []float64 = buf_2

Now as I understand it I have a copy of the reference but the same
underlying data.

If I now append to say buf_1.
curr_buffer = append(curr_buffer, data...)

After the append the length of curr_buffer is len(data) but the length of
buf_1 is zero.
As the capacity was sufficient to hold the data I didn't expect a
reassignment.

I also tried this without the local variables by splitting the function and
passing the references - same result.

I then tried pointers to references which worked, but that was expected.

Can someone enlighten me on the mechanism going on here please.

Thanks
bob

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Search Discussions

  • Matthew Kane at Nov 1, 2013 at 12:42 pm
    Remember that a slice is not just a pointer to a backing array; when you
    make a copy like you have here, you are making a copy of the whole slice
    header, including its capacity and length fields. When you modify
    curr_buffer, you aren't modifying the capacity and length bits of buf_1 or
    buf_2.



    On Fri, Nov 1, 2013 at 4:37 AM, wrote:

    Hi

    I think I must be misunderstanding something.

    I have 2 buffers to double buffer data.
    They are created with zero length and capacity to hold the data that will
    be written to them.
    buf_1 = make ([]float64, 0, n)
    buf_2 = make ([]float64, 0, n)

    I then select which buffer to use by assigning it to a local variable.
    var curr_buffer []float64 = buf_1
    var next_buffer []float64 = buf_2

    Now as I understand it I have a copy of the reference but the same
    underlying data.

    If I now append to say buf_1.
    curr_buffer = append(curr_buffer, data...)

    After the append the length of curr_buffer is len(data) but the length of
    buf_1 is zero.
    As the capacity was sufficient to hold the data I didn't expect a
    reassignment.

    I also tried this without the local variables by splitting the function
    and passing the references - same result.

    I then tried pointers to references which worked, but that was expected.

    Can someone enlighten me on the mechanism going on here please.

    Thanks
    bob

    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.


    --
    matt kane
    twitter: the_real_mkb / nynexrepublic
    http://hydrogenproject.com

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Jan Mercl at Nov 1, 2013 at 12:43 pm

    On Fri, Nov 1, 2013 at 9:37 AM, wrote:
    Can someone enlighten me on the mechanism going on here please.
    Slice is a simple value type - a small struct. If you work on a copy
    (of the slice value), the original (slice value) is not and cannot be
    affected in any way whatsoever.

    -j

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Bobcowdery at Nov 1, 2013 at 1:46 pm
    Ok, I understand. So one has to be very careful when passing references
    around because the underlying array will get changed but unless the
    reference is copied back to the original reference the change won't be
    reflected.if the size or capacity change. Are pointers to references then
    the only way to do this type of manipulation?

    bob
    On Friday, November 1, 2013 8:37:29 AM UTC, bobco...@gmail.com wrote:

    Hi

    I think I must be misunderstanding something.

    I have 2 buffers to double buffer data.
    They are created with zero length and capacity to hold the data that will
    be written to them.
    buf_1 = make ([]float64, 0, n)
    buf_2 = make ([]float64, 0, n)

    I then select which buffer to use by assigning it to a local variable.
    var curr_buffer []float64 = buf_1
    var next_buffer []float64 = buf_2

    Now as I understand it I have a copy of the reference but the same
    underlying data.

    If I now append to say buf_1.
    curr_buffer = append(curr_buffer, data...)

    After the append the length of curr_buffer is len(data) but the length of
    buf_1 is zero.
    As the capacity was sufficient to hold the data I didn't expect a
    reassignment.

    I also tried this without the local variables by splitting the function
    and passing the references - same result.

    I then tried pointers to references which worked, but that was expected.

    Can someone enlighten me on the mechanism going on here please.

    Thanks
    bob
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Matthew Kane at Nov 1, 2013 at 2:02 pm
    You don't need the buf_1 and buf_2 variables at all.

    var curr_buffer = make([]float64, 0, n)
    var next_buffer = make([]float64, 0, n)

    func swap() {
         curr_buffer, next_buffer = next_buffer[0:0], curr_buffer[0:0] // keeps
    cap, resets len to 0
    }

    On Fri, Nov 1, 2013 at 9:46 AM, wrote:

    Ok, I understand. So one has to be very careful when passing references
    around because the underlying array will get changed but unless the
    reference is copied back to the original reference the change won't be
    reflected.if the size or capacity change. Are pointers to references then
    the only way to do this type of manipulation?

    bob

    On Friday, November 1, 2013 8:37:29 AM UTC, bobco...@gmail.com wrote:

    Hi

    I think I must be misunderstanding something.

    I have 2 buffers to double buffer data.
    They are created with zero length and capacity to hold the data that will
    be written to them.
    buf_1 = make ([]float64, 0, n)
    buf_2 = make ([]float64, 0, n)

    I then select which buffer to use by assigning it to a local variable.
    var curr_buffer []float64 = buf_1
    var next_buffer []float64 = buf_2

    Now as I understand it I have a copy of the reference but the same
    underlying data.

    If I now append to say buf_1.
    curr_buffer = append(curr_buffer, data...)

    After the append the length of curr_buffer is len(data) but the length of
    buf_1 is zero.
    As the capacity was sufficient to hold the data I didn't expect a
    reassignment.

    I also tried this without the local variables by splitting the function
    and passing the references - same result.

    I then tried pointers to references which worked, but that was expected.

    Can someone enlighten me on the mechanism going on here please.

    Thanks
    bob
    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.


    --
    matt kane
    twitter: the_real_mkb / nynexrepublic
    http://hydrogenproject.com

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Chris dollin at Nov 1, 2013 at 2:09 pm

    On 1 November 2013 13:46, wrote:

    Ok, I understand. So one has to be very careful when passing references
    around
    (yes)

    because the underlying array will get changed but unless the reference is
    copied back to the original reference the change won't be reflected.
    That's right: the len and cap are parts of the /slice value/, not of the
    /underlying array/.

    if the size or capacity change. Are pointers to references then the only
    way to do this type of manipulation?
    You can return the new slice as a result, which is what I'd do
    unless there was good reason not to.

    Chris

    --
    Chris "allusive" Dollin

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Rob Pike at Nov 1, 2013 at 2:31 pm
    blog.golang.org/slices

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Bobcowdery at Nov 1, 2013 at 5:40 pm
    The buffers are part of my type state and get progressively filled from
    another go routine, one of many. It may be I'm over egging it but don't
    want to change the design. I've figured how to do it without using
    pointers. My actual problem is corruption which gets progressively worse
    the more parallel paths I have through my pipline which is 6 main go
    routines. I'm going through making sure I understand the implications and
    I'm much clearer on references now but I've still failed to find any where
    I'm doing it wrong so far.
    I do pass references around a lot, return them a lot and between go
    routines the message format uses the interface{} to hold references as
    these messages are of different types. In all case except the one I asked
    the question about the slices are pre-allocated to correct length in a
    memory allocator which re-cycles blocks (necessary because its real-time).

    Am I right in thinking that as long as slice size does not change I can
    throw them around, in and out of empty interface etc with impunity?

    Thanks for the help so far. The Wiki ref was a good read.

    bob
    On Friday, November 1, 2013 8:37:29 AM UTC, bobco...@gmail.com wrote:

    Hi

    I think I must be misunderstanding something.

    I have 2 buffers to double buffer data.
    They are created with zero length and capacity to hold the data that will
    be written to them.
    buf_1 = make ([]float64, 0, n)
    buf_2 = make ([]float64, 0, n)

    I then select which buffer to use by assigning it to a local variable.
    var curr_buffer []float64 = buf_1
    var next_buffer []float64 = buf_2

    Now as I understand it I have a copy of the reference but the same
    underlying data.

    If I now append to say buf_1.
    curr_buffer = append(curr_buffer, data...)

    After the append the length of curr_buffer is len(data) but the length of
    buf_1 is zero.
    As the capacity was sufficient to hold the data I didn't expect a
    reassignment.

    I also tried this without the local variables by splitting the function
    and passing the references - same result.

    I then tried pointers to references which worked, but that was expected.

    Can someone enlighten me on the mechanism going on here please.

    Thanks
    bob
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Dave Cheney at Nov 2, 2013 at 6:39 am

    On Sat, Nov 2, 2013 at 4:40 AM, wrote:
    The buffers are part of my type state and get progressively filled from
    another go routine, one of many.
    What does the race detector say about this ? From your description I
    would be concerned about data races.

    It may be I'm over egging it but don't want
    to change the design. I've figured how to do it without using pointers. My
    actual problem is corruption which gets progressively worse the more
    parallel paths I have through my pipline which is 6 main go routines. I'm
    going through making sure I understand the implications and I'm much clearer
    on references now but I've still failed to find any where I'm doing it wrong
    so far.
    I do pass references around a lot, return them a lot and between go routines
    the message format uses the interface{} to hold references as these messages
    are of different types. In all case except the one I asked the question
    about the slices are pre-allocated to correct length in a memory allocator
    which re-cycles blocks (necessary because its real-time).

    Am I right in thinking that as long as slice size does not change I can
    throw them around, in and out of empty interface etc with impunity?

    Thanks for the help so far. The Wiki ref was a good read.


    bob
    On Friday, November 1, 2013 8:37:29 AM UTC, bobco...@gmail.com wrote:

    Hi

    I think I must be misunderstanding something.

    I have 2 buffers to double buffer data.
    They are created with zero length and capacity to hold the data that will
    be written to them.
    buf_1 = make ([]float64, 0, n)
    buf_2 = make ([]float64, 0, n)

    I then select which buffer to use by assigning it to a local variable.
    var curr_buffer []float64 = buf_1
    var next_buffer []float64 = buf_2

    Now as I understand it I have a copy of the reference but the same
    underlying data.

    If I now append to say buf_1.
    curr_buffer = append(curr_buffer, data...)

    After the append the length of curr_buffer is len(data) but the length of
    buf_1 is zero.
    As the capacity was sufficient to hold the data I didn't expect a
    reassignment.

    I also tried this without the local variables by splitting the function
    and passing the references - same result.

    I then tried pointers to references which worked, but that was expected.

    Can someone enlighten me on the mechanism going on here please.

    Thanks
    bob
    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Bobcowdery at Nov 2, 2013 at 6:34 pm
    Looks like the race detector does not work with 32bit Windows which I'm
    currently running. I think I've run out of possibilities with malusing data
    types so I will turn to concurrency now although I was very careful to
    avoid race conditions and have checked the code so many times now.
    On Friday, November 1, 2013 8:37:29 AM UTC, bobco...@gmail.com wrote:

    Hi

    I think I must be misunderstanding something.

    I have 2 buffers to double buffer data.
    They are created with zero length and capacity to hold the data that will
    be written to them.
    buf_1 = make ([]float64, 0, n)
    buf_2 = make ([]float64, 0, n)

    I then select which buffer to use by assigning it to a local variable.
    var curr_buffer []float64 = buf_1
    var next_buffer []float64 = buf_2

    Now as I understand it I have a copy of the reference but the same
    underlying data.

    If I now append to say buf_1.
    curr_buffer = append(curr_buffer, data...)

    After the append the length of curr_buffer is len(data) but the length of
    buf_1 is zero.
    As the capacity was sufficient to hold the data I didn't expect a
    reassignment.

    I also tried this without the local variables by splitting the function
    and passing the references - same result.

    I then tried pointers to references which worked, but that was expected.

    Can someone enlighten me on the mechanism going on here please.

    Thanks
    bob
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Tamás Gulácsi at Nov 3, 2013 at 6:49 am
    Maybe I don't understand the problem correctly, but if buf_1 does not have enough capacity for data, then the slice returned by append will be different than the one you wanted to append to - although the new slice contains the olds data, too.

    Thus you need to push the actual value to buf_1 and buf_2 back to the global vars.

    If you do this from different goroutines, then some guard is needed for global var update.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Bobcowdery at Nov 3, 2013 at 5:22 pm
    Those buffers are in one go routine so no conflict. Buffers passed between
    go routines are allocated from the pool by the sender and freed back to the
    pool by the receiver. I did initially have some bad corruption going on due
    to not copying data in certain places where partial data went to more than
    one go routine and having references into slices which were then recycled.
    After fixing those issues I was left with some strangeness in the data
    which I just can't find the cause of. I've checked the return from all the
    copy statements and the expected amount of data is always copied.

    I should probably explain that this is a software radio implementation and
    I have random clicking and distortion in the output. The hardware can run
    up to 8 receivers and if I enable more than one which creates parallel
    paths in the program for the signal processing the corruption gets worse to
    the point of no discernible signals. The hardware is not at fault because I
    have other software that works correctly. I'm also pretty sure my
    application code is correct as I've implemented this before in other
    languages. There is a lot going on in this app and clearly I've messed up
    somewhere but I'm running out of debug approaches. I'm using the latest go
    beta and assuming the language is solid enough not to be causing this.

    Any suggested approaches very welcome.

    bob
    On Friday, November 1, 2013 8:37:29 AM UTC, bobco...@gmail.com wrote:

    Hi

    I think I must be misunderstanding something.

    I have 2 buffers to double buffer data.
    They are created with zero length and capacity to hold the data that will
    be written to them.
    buf_1 = make ([]float64, 0, n)
    buf_2 = make ([]float64, 0, n)

    I then select which buffer to use by assigning it to a local variable.
    var curr_buffer []float64 = buf_1
    var next_buffer []float64 = buf_2

    Now as I understand it I have a copy of the reference but the same
    underlying data.

    If I now append to say buf_1.
    curr_buffer = append(curr_buffer, data...)

    After the append the length of curr_buffer is len(data) but the length of
    buf_1 is zero.
    As the capacity was sufficient to hold the data I didn't expect a
    reassignment.

    I also tried this without the local variables by splitting the function
    and passing the references - same result.

    I then tried pointers to references which worked, but that was expected.

    Can someone enlighten me on the mechanism going on here please.

    Thanks
    bob
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedNov 1, '13 at 12:29p
activeNov 3, '13 at 5:22p
posts12
users7
websitegolang.org

People

Translate

site design / logo © 2022 Grokbase