FAQ
Hi all,

This is my first golang-nuts post and first real Go project, so please be
gentle.

I am currently in the process of writing a cgo wrapper for a C library that
uses a lot of C99 flexible array members[0], and I'm basically wondering
what is the prescribed way to successfully write such a wrapper.

Here is the overview of what I'm trying to do:

Given a struct with a flexible array member, e.g.:

typedef struct addr_t addr_t;
struct addr_t {
         uint8_t len;
         uint8_t data[];
} __attribute__((packed)) __attribute__((aligned(1)));

I want to be able to write a cgo wrapper that allocates memory for an
addr_t object, in such a way that I can put values in the "flexible" data
portion of the struct.

In my attempts I have tried dynamically allocating the memory and casting
it to a *addr_t (much like you would in C), i.e.:

a := (*C.addr_t)(C.malloc(C.size_t(unsafe.Sizeof(C.addr_t{}) +
C.MAX_ADDR_LEN)))

This doesn't appear to do the trick. I can subsequently assign to a.len,
but indexing into data[] gives me an "index out of range" panic. I'd also
be curious how to accomplish my objectives with statically allocated memory.

I have coded up and attached what I consider to be a decent test case,
using the above dynamic allocation approach. Just do the following to
reproduce:

1. Decompress (e.g., tar -zxvf cgo-test.tar.gz)
2. Install "main" package (e.g., cd cgo-test; go install main)
3. Run (e.g., ./bin/main)

You should see an addr_t value being printed out by Go (using %+v): "a:
&{len:3 data:[]}". Notice that the data portion appears to not have any
storage. After that, you should see a runtime panic (index out of range),
because the code is trying to poke some value in that data portion.

Thanks for any advice.

         Brian

[0] http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.dui0472c/BABECHJI.html
(or just google the term "flexible array member")



--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Search Discussions

  • Kyle Lemons at May 24, 2013 at 2:53 am
    (warning: the following code is produced in a facility which also handles
    nuts, and may not be safe to eat)

    Allocating it in Go is a fight that I wouldn't wish on anyone. I think
    it's simpler to make a C function that allocates it:

    addr_t *alloc_addr_t(int len) {
       addr_t *ret = (addr_t*)malloc(sizeof(addr_t) + len*sizeof(uint8_t));
       ret.len = len;
       return ret;
    }

    (which can go in the comment before your import "C")

    The following is also pretty straightforward:

    func fromC(raw C.addr_t) []uint8 {
       slice := reflect.SliceHeader{
         Data: uintptr(unsafe.Pointer(raw.data)),
         Len: int(raw.len),
         Cap: int(raw.len),
       }
       return *(*[]uint8)(unsafe.Pointer(&slice))
    }

    So, if you need to pass one somewhere:

    raw := C.alloc_addr_t(C.int(count))
    defer C.free(raw)
    slice := fromC(raw)
    for i := range raw {
       raw[i] = uint8(i)
    }
    C.some_function_that_takes_addr_t(raw)

    Or if you get one:

    raw := C.some_function_that_returns_addr_t()
    defer C.free(raw)
    for i, v := range fromC(raw) {
       fmt.Printf("v[%d] = %d\n", i, v)
    }



    On Thu, May 23, 2013 at 5:31 PM, Brian G. Merrell wrote:

    Hi all,

    This is my first golang-nuts post and first real Go project, so please be
    gentle.

    I am currently in the process of writing a cgo wrapper for a C library
    that uses a lot of C99 flexible array members[0], and I'm basically
    wondering what is the prescribed way to successfully write such a wrapper.

    Here is the overview of what I'm trying to do:

    Given a struct with a flexible array member, e.g.:

    typedef struct addr_t addr_t;
    struct addr_t {
    uint8_t len;
    uint8_t data[];
    } __attribute__((packed)) __attribute__((aligned(1)));

    I want to be able to write a cgo wrapper that allocates memory for an
    addr_t object, in such a way that I can put values in the "flexible" data
    portion of the struct.

    In my attempts I have tried dynamically allocating the memory and casting
    it to a *addr_t (much like you would in C), i.e.:

    a := (*C.addr_t)(C.malloc(C.size_t(unsafe.Sizeof(C.addr_t{}) +
    C.MAX_ADDR_LEN)))

    This doesn't appear to do the trick. I can subsequently assign to a.len,
    but indexing into data[] gives me an "index out of range" panic. I'd also
    be curious how to accomplish my objectives with statically allocated memory.

    I have coded up and attached what I consider to be a decent test case,
    using the above dynamic allocation approach. Just do the following to
    reproduce:

    1. Decompress (e.g., tar -zxvf cgo-test.tar.gz)
    2. Install "main" package (e.g., cd cgo-test; go install main)
    3. Run (e.g., ./bin/main)

    You should see an addr_t value being printed out by Go (using %+v): "a:
    &{len:3 data:[]}". Notice that the data portion appears to not have any
    storage. After that, you should see a runtime panic (index out of range),
    because the code is trying to poke some value in that data portion.

    Thanks for any advice.

    Brian

    [0]
    http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.dui0472c/BABECHJI.html(or just google the term "flexible array member")



    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Kyle Lemons at May 24, 2013 at 2:55 am

    On Thu, May 23, 2013 at 7:53 PM, Kyle Lemons wrote:

    (warning: the following code is produced in a facility which also handles
    nuts, and may not be safe to eat)

    Allocating it in Go is a fight that I wouldn't wish on anyone. I think
    it's simpler to make a C function that allocates it:

    addr_t *alloc_addr_t(int len) {
    addr_t *ret = (addr_t*)malloc(sizeof(addr_t) + len*sizeof(uint8_t));
    ret.len = len;
    return ret;
    }

    (which can go in the comment before your import "C")

    The following is also pretty straightforward:

    func fromC(raw C.addr_t) []uint8 {
    slice := reflect.SliceHeader{
    Data: uintptr(unsafe.Pointer(raw.data)),
    Len: int(raw.len),
    Cap: int(raw.len),
    }
    return *(*[]uint8)(unsafe.Pointer(&slice))
    }

    So, if you need to pass one somewhere:

    raw := C.alloc_addr_t(C.int(count))
    defer C.free(raw)
    slice := fromC(raw)
    for i := range raw {
    raw[i] = uint8(i)
    }
    C.some_function_that_takes_addr_t(raw)
    d'oh, that should be "range slice" and "slice[i] ="

    Or if you get one:

    raw := C.some_function_that_returns_addr_t()
    defer C.free(raw)
    for i, v := range fromC(raw) {
    fmt.Printf("v[%d] = %d\n", i, v)
    }



    On Thu, May 23, 2013 at 5:31 PM, Brian G. Merrell wrote:

    Hi all,

    This is my first golang-nuts post and first real Go project, so please be
    gentle.

    I am currently in the process of writing a cgo wrapper for a C library
    that uses a lot of C99 flexible array members[0], and I'm basically
    wondering what is the prescribed way to successfully write such a wrapper.

    Here is the overview of what I'm trying to do:

    Given a struct with a flexible array member, e.g.:

    typedef struct addr_t addr_t;
    struct addr_t {
    uint8_t len;
    uint8_t data[];
    } __attribute__((packed)) __attribute__((aligned(1)));

    I want to be able to write a cgo wrapper that allocates memory for an
    addr_t object, in such a way that I can put values in the "flexible" data
    portion of the struct.

    In my attempts I have tried dynamically allocating the memory and casting
    it to a *addr_t (much like you would in C), i.e.:

    a := (*C.addr_t)(C.malloc(C.size_t(unsafe.Sizeof(C.addr_t{}) +
    C.MAX_ADDR_LEN)))

    This doesn't appear to do the trick. I can subsequently assign to a.len,
    but indexing into data[] gives me an "index out of range" panic. I'd also
    be curious how to accomplish my objectives with statically allocated memory.

    I have coded up and attached what I consider to be a decent test case,
    using the above dynamic allocation approach. Just do the following to
    reproduce:

    1. Decompress (e.g., tar -zxvf cgo-test.tar.gz)
    2. Install "main" package (e.g., cd cgo-test; go install main)
    3. Run (e.g., ./bin/main)

    You should see an addr_t value being printed out by Go (using %+v): "a:
    &{len:3 data:[]}". Notice that the data portion appears to not have any
    storage. After that, you should see a runtime panic (index out of range),
    because the code is trying to poke some value in that data portion.

    Thanks for any advice.

    Brian

    [0]
    http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.dui0472c/BABECHJI.html(or just google the term "flexible array member")



    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Brian G. Merrell at May 24, 2013 at 8:19 pm
    Thanks for the tips, Kyle. They were very helpful. As far as I can tell,
    I didn't actually need the alloc_addr_t to allocate the memory. My
    original Go-based malloc approach appears to work, but what I did need was
    the fromC function to build the slice. Is there anything wrong with this
    approach below? I'd still be curious how to do this with statically
    allocated memory (i.e., not using malloc).

    type Addr string // an address is just some uint8s separated by spaces,
    e.g., "1 2 1"

    /* Take an Addr and return a slice of atoi'd uint8s */
    func (a *Addr) Uint8Fields() []uint8 {
    fields := strings.Fields(string(*a))
    var intFields []uint8 = make([]uint8, len(fields))
    for i, char := range fields {
    n, err := strconv.Atoi(char)
    if err != nil {
    return make([]uint8, 0)
    }
    intFields[i] = uint8(n)
    }
    return intFields
    }

    /* Create a custom slice backed by the addr_t's data */
    func fromC(raw_addr *C.addr_t) []uint8 {
    slice := reflect.SliceHeader{
    Data: uintptr(unsafe.Pointer(&raw_addr.data)),
    Len: int(raw_addr.len),
    Cap: int(raw_addr.len),
    }
    return *(*[]uint8)(unsafe.Pointer(&slice))
    }

    /* Public function that takes a Go Addr, turn it into a C addr_t, and pass
    that on to a C function */
    func AddrWrap(addr Addr) {
    var split_addr []uint8 = addr.Uint8Fields()
    raw_addr := (*C.addr_t)(C.malloc(C.size_t(unsafe.Sizeof(C.addr_t{}) +
    C.MAX_ADDR_LEN)))
    defer C.free(unsafe.Pointer(raw_addr))
    raw_addr.len = C.uint8_t(len(split_addr))
    slice := fromC(raw_addr)
    for i := range slice {
    slice[i] = split_addr[i]
    }
    C.print_addr(raw_addr) // Print the addr_t len and data from C
    }

    Thanks again,
    Brian
    On Thursday, May 23, 2013 8:54:59 PM UTC-6, Kyle Lemons wrote:

    On Thu, May 23, 2013 at 7:53 PM, Kyle Lemons <kev...@google.com<javascript:>
    wrote:
    (warning: the following code is produced in a facility which also handles
    nuts, and may not be safe to eat)

    Allocating it in Go is a fight that I wouldn't wish on anyone. I think
    it's simpler to make a C function that allocates it:

    addr_t *alloc_addr_t(int len) {
    addr_t *ret = (addr_t*)malloc(sizeof(addr_t) + len*sizeof(uint8_t));
    ret.len = len;
    return ret;
    }

    (which can go in the comment before your import "C")

    The following is also pretty straightforward:

    func fromC(raw C.addr_t) []uint8 {
    slice := reflect.SliceHeader{
    Data: uintptr(unsafe.Pointer(raw.data)),
    Len: int(raw.len),
    Cap: int(raw.len),
    }
    return *(*[]uint8)(unsafe.Pointer(&slice))
    }

    So, if you need to pass one somewhere:

    raw := C.alloc_addr_t(C.int(count))
    defer C.free(raw)
    slice := fromC(raw)
    for i := range raw {
    raw[i] = uint8(i)
    }
    C.some_function_that_takes_addr_t(raw)
    d'oh, that should be "range slice" and "slice[i] ="

    Or if you get one:

    raw := C.some_function_that_returns_addr_t()
    defer C.free(raw)
    for i, v := range fromC(raw) {
    fmt.Printf("v[%d] = %d\n", i, v)
    }




    On Thu, May 23, 2013 at 5:31 PM, Brian G. Merrell <bgme...@gmail.com<javascript:>
    wrote:
    Hi all,

    This is my first golang-nuts post and first real Go project, so please
    be gentle.

    I am currently in the process of writing a cgo wrapper for a C library
    that uses a lot of C99 flexible array members[0], and I'm basically
    wondering what is the prescribed way to successfully write such a wrapper.

    Here is the overview of what I'm trying to do:

    Given a struct with a flexible array member, e.g.:

    typedef struct addr_t addr_t;
    struct addr_t {
    uint8_t len;
    uint8_t data[];
    } __attribute__((packed)) __attribute__((aligned(1)));

    I want to be able to write a cgo wrapper that allocates memory for an
    addr_t object, in such a way that I can put values in the "flexible" data
    portion of the struct.

    In my attempts I have tried dynamically allocating the memory and
    casting it to a *addr_t (much like you would in C), i.e.:

    a := (*C.addr_t)(C.malloc(C.size_t(unsafe.Sizeof(C.addr_t{}) +
    C.MAX_ADDR_LEN)))

    This doesn't appear to do the trick. I can subsequently assign to
    a.len, but indexing into data[] gives me an "index out of range" panic.
    I'd also be curious how to accomplish my objectives with statically
    allocated memory.

    I have coded up and attached what I consider to be a decent test case,
    using the above dynamic allocation approach. Just do the following to
    reproduce:

    1. Decompress (e.g., tar -zxvf cgo-test.tar.gz)
    2. Install "main" package (e.g., cd cgo-test; go install main)
    3. Run (e.g., ./bin/main)

    You should see an addr_t value being printed out by Go (using %+v): "a:
    &{len:3 data:[]}". Notice that the data portion appears to not have any
    storage. After that, you should see a runtime panic (index out of range),
    because the code is trying to poke some value in that data portion.

    Thanks for any advice.

    Brian

    [0]
    http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.dui0472c/BABECHJI.html(or just google the term "flexible array member")



    --
    You received this message because you are subscribed to the Google
    Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send
    an email to golang-nuts...@googlegroups.com <javascript:>.
    For more options, visit https://groups.google.com/groups/opt_out.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Kyle Lemons at May 24, 2013 at 10:25 pm

    On Fri, May 24, 2013 at 1:19 PM, Brian G. Merrell wrote:

    Thanks for the tips, Kyle. They were very helpful. As far as I can tell,
    I didn't actually need the alloc_addr_t to allocate the memory.
    No, it's not necessary, but I think it's cleaner than sifting through the
    layers of unsafe.

    My original Go-based malloc approach appears to work, but what I did need
    was the fromC function to build the slice. Is there anything wrong with
    this approach below?
    I'd still be curious how to do this with statically allocated memory
    (i.e., not using malloc).
    I don't recommend it. You could make a [N]byte where N is the size of what
    you want, but you may baffle the compiler's ability to decide whether to
    put it on the stack or the heap and/or the garbage collector.

    type Addr string // an address is just some uint8s separated by spaces,
    e.g., "1 2 1"

    /* Take an Addr and return a slice of atoi'd uint8s */
    func (a *Addr) Uint8Fields() []uint8 {
    fields := strings.Fields(string(*a))
    var intFields []uint8 = make([]uint8, len(fields))
    for i, char := range fields {
    n, err := strconv.Atoi(char)
    if err != nil {
    return make([]uint8, 0)
    }
    intFields[i] = uint8(n)
    }
    return intFields
    }

    /* Create a custom slice backed by the addr_t's data */
    func fromC(raw_addr *C.addr_t) []uint8 {
    slice := reflect.SliceHeader{
    Data: uintptr(unsafe.Pointer(&raw_addr.data)),
    Len: int(raw_addr.len),
    Cap: int(raw_addr.len),
    }
    return *(*[]uint8)(unsafe.Pointer(&slice))
    }

    /* Public function that takes a Go Addr, turn it into a C addr_t, and pass
    that on to a C function */
    func AddrWrap(addr Addr) {
    var split_addr []uint8 = addr.Uint8Fields()
    raw_addr := (*C.addr_t)(C.malloc(C.size_t(unsafe.Sizeof(C.addr_t{}) +
    C.MAX_ADDR_LEN)))
    defer C.free(unsafe.Pointer(raw_addr))
    raw_addr.len = C.uint8_t(len(split_addr))
    slice := fromC(raw_addr)
    for i := range slice {
    slice[i] = split_addr[i]
    }
    C.print_addr(raw_addr) // Print the addr_t len and data from C
    }

    Thanks again,
    Brian
    On Thursday, May 23, 2013 8:54:59 PM UTC-6, Kyle Lemons wrote:
    On Thu, May 23, 2013 at 7:53 PM, Kyle Lemons wrote:

    (warning: the following code is produced in a facility which also
    handles nuts, and may not be safe to eat)

    Allocating it in Go is a fight that I wouldn't wish on anyone. I think
    it's simpler to make a C function that allocates it:

    addr_t *alloc_addr_t(int len) {
    addr_t *ret = (addr_t*)malloc(sizeof(addr_t) + len*sizeof(uint8_t));
    ret.len = len;
    return ret;
    }

    (which can go in the comment before your import "C")

    The following is also pretty straightforward:

    func fromC(raw C.addr_t) []uint8 {
    slice := reflect.SliceHeader{
    Data: uintptr(unsafe.Pointer(raw.**data)),
    Len: int(raw.len),
    Cap: int(raw.len),
    }
    return *(*[]uint8)(unsafe.Pointer(&**slice))
    }

    So, if you need to pass one somewhere:

    raw := C.alloc_addr_t(C.int(count))
    defer C.free(raw)
    slice := fromC(raw)
    for i := range raw {
    raw[i] = uint8(i)
    }
    C.some_function_that_takes_**addr_t(raw)
    d'oh, that should be "range slice" and "slice[i] ="

    Or if you get one:

    raw := C.some_function_that_returns_**addr_t()
    defer C.free(raw)
    for i, v := range fromC(raw) {
    fmt.Printf("v[%d] = %d\n", i, v)
    }



    On Thu, May 23, 2013 at 5:31 PM, Brian G. Merrell wrote:

    Hi all,

    This is my first golang-nuts post and first real Go project, so please
    be gentle.

    I am currently in the process of writing a cgo wrapper for a C library
    that uses a lot of C99 flexible array members[0], and I'm basically
    wondering what is the prescribed way to successfully write such a wrapper.

    Here is the overview of what I'm trying to do:

    Given a struct with a flexible array member, e.g.:

    typedef struct addr_t addr_t;
    struct addr_t {
    uint8_t len;
    uint8_t data[];
    } __attribute__((packed)) __attribute__((aligned(1)));

    I want to be able to write a cgo wrapper that allocates memory for an
    addr_t object, in such a way that I can put values in the "flexible" data
    portion of the struct.

    In my attempts I have tried dynamically allocating the memory and
    casting it to a *addr_t (much like you would in C), i.e.:

    a := (*C.addr_t)(C.malloc(C.size_t(**unsafe.Sizeof(C.addr_t{}) +
    C.MAX_ADDR_LEN)))

    This doesn't appear to do the trick. I can subsequently assign to
    a.len, but indexing into data[] gives me an "index out of range" panic.
    I'd also be curious how to accomplish my objectives with statically
    allocated memory.

    I have coded up and attached what I consider to be a decent test case,
    using the above dynamic allocation approach. Just do the following to
    reproduce:

    1. Decompress (e.g., tar -zxvf cgo-test.tar.gz)
    2. Install "main" package (e.g., cd cgo-test; go install main)
    3. Run (e.g., ./bin/main)

    You should see an addr_t value being printed out by Go (using %+v): "a:
    &{len:3 data:[]}". Notice that the data portion appears to not have any
    storage. After that, you should see a runtime panic (index out of range),
    because the code is trying to poke some value in that data portion.

    Thanks for any advice.

    Brian

    [0] http://infocenter.arm.com/**help/index.jsp?topic=/com.arm.**
    doc.dui0472c/BABECHJI.html<http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.dui0472c/BABECHJI.html>(or just google the term "flexible array member")



    --
    You received this message because you are subscribed to the Google
    Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send
    an email to golang-nuts...@**googlegroups.com.

    For more options, visit https://groups.google.com/**groups/opt_out<https://groups.google.com/groups/opt_out>
    .

    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedMay 24, '13 at 2:05a
activeMay 24, '13 at 10:25p
posts5
users2
websitegolang.org

2 users in discussion

Kyle Lemons: 3 posts Brian G. Merrell: 2 posts

People

Translate

site design / logo © 2021 Grokbase