FAQ
Hi,

This is probably a case of premature optimization, but I'd like to ask
just out of curiosity. Cgo provides the C.CString function for
converting a Go string into a null-terminated *C.char. Once that
string is no longer needed, it must be freed. This allocate/free
routine can get a little expensive when multiple strings need to be
converted in this manner.

I wrote the following function that accepts a list of strings and
concatenates them, separated by a single null, into a Go byte slice. A
*C.char is returned back to the caller for each input string.
According to my benchmarks, this function is about twice as fast as
using C.CString/C.free. What I'd like to know is do I need to keep a
reference to the []byte in order to prevent the GC from releasing that
memory, or is having at least one C.char pointer into the underlying
array sufficient?

func cstr(str ...string) []*C.char {
n := len(str)
for _, s := range str {
n += len(s)
}
buf := make([]byte, n)
ret := make([]*C.char, len(str))
for i, s := range str {
ret[i] = (*C.char)(unsafe.Pointer(&buf[0]))
buf = buf[copy(buf, s)+1:]
}
return ret
}

- Max

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Search Discussions

  • Minux at Feb 4, 2013 at 7:24 pm

    On Tue, Feb 5, 2013 at 2:37 AM, Maxim Khitrov wrote:

    This is probably a case of premature optimization, but I'd like to ask
    just out of curiosity. Cgo provides the C.CString function for
    converting a Go string into a null-terminated *C.char. Once that
    string is no longer needed, it must be freed. This allocate/free
    routine can get a little expensive when multiple strings need to be
    converted in this manner.

    I wrote the following function that accepts a list of strings and
    concatenates them, separated by a single null, into a Go byte slice. A
    *C.char is returned back to the caller for each input string.
    According to my benchmarks, this function is about twice as fast as
    using C.CString/C.free. What I'd like to know is do I need to keep a
    reference to the []byte in order to prevent the GC from releasing that
    memory, or is having at least one C.char pointer into the underlying
    array sufficient?
    if you only provide the string to C code, you can use C.malloc, then you
    don't
    need to keep the slice referenced in Go code to prevent the whole byte array
    backing the buf collected.

    Regarding your original question, IMO you shouldn't depend on the fact that
    keeping the *C.char could pin the whole buf so your best bet is to retain a
    reference to the returned slice in Go code (or use C.malloc).
    func cstr(str ...string) []*C.char {
    n := len(str)
    for _, s := range str {
    n += len(s)
    }
    buf := make([]byte, n)
    ret := make([]*C.char, len(str))
    for i, s := range str {
    ret[i] = (*C.char)(unsafe.Pointer(&buf[0]))
    buf = buf[copy(buf, s)+1:]
    }
    return ret
    }
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Maxim Khitrov at Feb 4, 2013 at 10:26 pm

    On Mon, Feb 4, 2013 at 2:24 PM, minux wrote:
    On Tue, Feb 5, 2013 at 2:37 AM, Maxim Khitrov wrote:

    This is probably a case of premature optimization, but I'd like to ask
    just out of curiosity. Cgo provides the C.CString function for
    converting a Go string into a null-terminated *C.char. Once that
    string is no longer needed, it must be freed. This allocate/free
    routine can get a little expensive when multiple strings need to be
    converted in this manner.

    I wrote the following function that accepts a list of strings and
    concatenates them, separated by a single null, into a Go byte slice. A
    *C.char is returned back to the caller for each input string.
    According to my benchmarks, this function is about twice as fast as
    using C.CString/C.free. What I'd like to know is do I need to keep a
    reference to the []byte in order to prevent the GC from releasing that
    memory, or is having at least one C.char pointer into the underlying
    array sufficient?
    if you only provide the string to C code, you can use C.malloc, then you
    don't
    need to keep the slice referenced in Go code to prevent the whole byte array
    backing the buf collected.

    Regarding your original question, IMO you shouldn't depend on the fact that
    keeping the *C.char could pin the whole buf so your best bet is to retain a
    reference to the returned slice in Go code (or use C.malloc).
    Tried doing the same thing with malloc and the performance was about
    the same as C.CString. Maybe it's just the cgo overhead, which goes
    away when using []byte. Think I'll just go back to using C.CString to
    avoid complicating the code. Thanks!

    - Max

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Minux at Feb 4, 2013 at 10:34 pm

    On Tue, Feb 5, 2013 at 6:26 AM, Maxim Khitrov wrote:
    Tried doing the same thing with malloc and the performance was about
    the same as C.CString. Maybe it's just the cgo overhead, which goes
    yes, it's the cgo overhead.
    we have to switch the stack to OS thread stack (this alone doesn't take
    long),
    and then tell the scheduler that we're going into C world, and be prepared
    to
    create or find other OS threads to run ready goroutines (this takes the
    major
    time).
    away when using []byte. Think I'll just go back to using C.CString to
    avoid complicating the code. Thanks!
    If this is really a bottleneck for you, i suggest you C.malloc a large chunk
    of memory and manage them yourself (note: if the C code that you're passing
    strings to never retain the string after it returns, you can continue to
    use your
    proposed solution in the original post, because when last reference of the
    slice disappears, your C function will already be returned; however, it the
    C
    code retains the pointers, you need a registry in Go for all those []*C.char
    slices)

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedFeb 4, '13 at 6:38p
activeFeb 4, '13 at 10:34p
posts4
users2
websitegolang.org

2 users in discussion

Minux: 2 posts Maxim Khitrov: 2 posts

People

Translate

site design / logo © 2022 Grokbase