[go-nuts] Static linking and thread-safe allocation

From knowing nothing about your problem except from what I learned from
looking at the stacktrace it would appear that your program failed to get
the number of processors on your machine once it left the jemalloc library.
Maybe trying printf'ing information at ../src/jemalloc.c:256?

Also, I just checked using this simple program and I can't get it to crash
by using the built-in malloc/free. Can you try running it to see if it
fails for you in the same way?

What system are you on? Did you check the docs for your malloc and find
that it isn't threadsafe? I've personally used malloc with pthreads, so I
know at least that works for a normal gcc/unix-y setup.

// main.go
package main

import (
         "fmt"
         "./cgo"
         "runtime"

         "sync"
)

func main() {
         ch := make(chan []byte, 1000)
         var waiter sync.WaitGroup
         waiter.Add(100)
         for i := 0; i < 100; i++ {
                 go allocater(waiter.Done, ch)
         }

         go func() {
                 waiter.Wait()
                 close(ch)
         }()
         for m := range ch {
                 m[2] = 'c'
                 fmt.Println(string(m[:3]))
                 cgo.Free(m)
         }
}

func allocater(done func(), dst chan<-[]byte) {
         defer done()
         // force a unique thread
         runtime.LockOSThread()

         for i := 0; i < 10; i++ {
                 m := cgo.Malloc(1024)
                 m[0] = 'a'
                 m[1] = 'b'
                 dst<-m
         }
}

// cgo/cgo.go
package cgo

// #include <stdlib.h>
import "C"

import (
         "reflect"
         "unsafe"
)

func Malloc(sz int) []byte {
         cbuf := C.malloc(C.size_t(sz))
         head := reflect.SliceHeader{uintptr(cbuf),sz,sz}

         return *(*[]byte)(unsafe.Pointer(&head))
}

func Free(data []byte) {
         C.free(unsafe.Pointer(&data[0]))
}

Thanks very much for your reply.

I must have misattributed this crash. I ran your example in my
environment and ensured that I was calling the same allocator used by
the C++ code and the example works. Thanks for helping me rule this
out.

Now I will have to try and figure out what the real problem is. I hope
valgrind works.

Not a problem. cgo takes a bit of getting used to but it generally works
out alright once you learn about the sharp edges. Let me know if you want
any more pointers.

I cracked my problem. The library I was calling isn't remotely
thread-safe. I was just mistaken.

I was under the impression gdb didn't work on go binaries, but it
seems it works fine on cgo binaries. So I just ran under gdb and saw
much more detail about the crashes, seeing deeper into the stack than
the "entersyscall()" in the runtime.

Thanks again for the pointers.

Not a problem. As a general rule I assume all C libraries are
single-threaded unless it specifically says so. I usually have one
goroutine per C library (with the thread locked) running some sort of loop
that serializes access to the library somehow (using channels, using a
callback provided during init, etc).