[go-nuts] Re: Bitten by Go 1.4 runtime changes

On Jul 8, 2015 9:50 AM, "Alexander Neumann" wrote:

Hah, I am happy to report that the just released Go 1.5 beta seems to fix
the problem.

Thanks to the Go team!
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

On Wednesday, July 8, 2015 at 7:05:29 PM UTC+2, bradfitz wrote:

You might just be getting lucky.

You still can't safely put pointers in uintptr. The GC will ignore them
and might free memory that you're still trying to use.
On Jul 8, 2015 9:50 AM, "Alexander Neumann" <an2...@googlemail.com
<javascript:>> wrote:

On Thu, Jul 9, 2015 at 1:21 AM, Alexander Neumann wrote:
I'm afraid you're right. Things just crash later now.

So if anybody has an idea on how to debug this, I would be happy to hear
about it.

On Thursday, July 9, 2015 at 12:20:32 PM UTC+2, Ian Lance Taylor wrote:
On Thu, Jul 9, 2015 at 1:21 AM, Alexander Neumann <an2...@googlemail.com
<javascript:>> wrote:
I'm not sure what you are trying to debug. You can not store pointer
values in variables of type uintptr. That was never safe. Now it is
even less safe.

On Wed, Jul 15, 2015 at 9:36 AM, Mateusz Czapliński wrote:
Umm; not exactly? If I understand correctly, in 1.4 you can store pointers
in uintptr, *as long as there are other references on heap*. (They must be
specifically on heap so that they're not moved together with pointee during
stack copying.) I'm trying to make sure to enforce that in my
https://github.com/akavel/winq package ("Package winq provides functions for
quick & dirty WinAPI calls, with easy errors capturing."), by keeping a
slice of syscall args (which sometimes are pointers) in []interface{} form
over a call to sub-function converting them to uintptr.

In 1.4 we had partial stack copying, so if you stored a pointer to a
stack address on the stack as a uintptr, the wrong thing would happen.
I agree that in 1.4, as well as 1.5, you can store a pointer to a heap
address as a uintptr and get away with it as long as there are other
pointers to that heap address. But it is not safe. We make
absolutely no promises that it will continue working in the future.

On Wed, Jul 15, 2015 at 12:12 PM wrote:

Do the uintptr and GC usage considerations need to be made more visible?

I've very recently started investigating the multi-platform FFI usage of
the syscall and unsafe packages. This thread has shown me I'm doing things
incorrectly. I haven't found anything as helpful as this thread in the
"first look" docs at

http://tip.golang.org/pkg/builtin/#uintptr
http://tip.godoc.org/unsafe#Pointer
http://tip.golang.org/ref/spec#Numeric_types
http://tip.golang.org/ref/spec#System_considerations

--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

On Wednesday, July 15, 2015 at 12:12:12 PM UTC-5, jon.f...@gmail.com wrote:

Do the uintptr and GC usage considerations need to be made more visible?

I've very recently started investigating the multi-platform FFI usage of
the syscall and unsafe packages. This thread has shown me I'm doing things
incorrectly. I haven't found anything as helpful as this thread in the
"first look" docs at

http://tip.golang.org/pkg/builtin/#uintptr
http://tip.godoc.org/unsafe#Pointer
http://tip.golang.org/ref/spec#Numeric_types
http://tip.golang.org/ref/spec#System_considerations

On Wed, Jul 15, 2015 at 12:37 PM Daniel Eloff wrote:

The gotchas around unsafe.Pointer and uintptr conversion should definitely
be documented better, with some examples of how GC and stack copying can
move things out from under you. It's an open-source project, so anyone can
write that up and do whatever is the blessed Go way of submitting a "pull
request". If someone does that, please post a draft of the changes here so
we can review it, contribute examples, etc.

However, the other part of the problem is that Go does not specify what
exactly is safe, other than to say "be sensible" which is deliberately a
vague at-your-own-risk kind of warning. The reason for that is what has
been safe is steadily shrinking. Stack copying and precise GC have made
these conversions a lot more fraught than they already were.

Some things *have* to keep working in a "sensible" Go. Converting to
uintptr and back to do address manipulation (e.g. the Go equivalent of
pointer[i]) if done without named variable uintptr intermediaries or
function calls must always work in a "sensible" world.

fooi := (*Foo)(unsafe.Pointer((uintptr)(unsafe.Pointer(p)) + i)) // Safe

fooi := (*Foo)(unsafe.Pointer((uintptr)(unsafe.Pointer(p)) +
some_function(i))) // Potentially unsafe

addr := (uintptr)(unsafe.Pointer(p))
fooi := (*Foo)(unsafe.Pointer(addr + i)) // Unsafe, even if you can
promise p points to the heap, and a strong reference to p exists, it can
still break in the future

Anything else is not guaranteed to be safe by Go. It is currently unsafe
unless you can prove that the object is on the heap and an anchored pointer
exists to it. And it won't be safe at all in the future if a moving GC is
introduced.


On Wednesday, July 15, 2015 at 12:12:12 PM UTC-5, jon.f...@gmail.com
wrote:
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

On Wednesday, July 15, 2015 at 8:18:55 PM UTC+2, Matt Harden wrote:
A workaround is to use unmanaged memory when passing pointers to/from
non-Go.

On Wed, Jul 15, 2015 at 12:37 PM Daniel Eloff <dan....@gmail.com
<javascript:>> wrote:

be documented better, with some examples of how GC and stack copying can
move things out from under you. It's an open-source project, so anyone can
write that up and do whatever is the blessed Go way of submitting a "pull
request". If someone does that, please post a draft of the changes here so
we can review it, contribute examples, etc.

However, the other part of the problem is that Go does not specify what
exactly is safe, other than to say "be sensible" which is deliberately a
vague at-your-own-risk kind of warning. The reason for that is what has
been safe is steadily shrinking. Stack copying and precise GC have made
these conversions a lot more fraught than they already were.

Some things *have* to keep working in a "sensible" Go. Converting to
uintptr and back to do address manipulation (e.g. the Go equivalent of
pointer[i]) if done without named variable uintptr intermediaries or
function calls must always work in a "sensible" world.

fooi := (*Foo)(unsafe.Pointer((uintptr)(unsafe.Pointer(p)) + i)) // Safe

fooi := (*Foo)(unsafe.Pointer((uintptr)(unsafe.Pointer(p)) +
some_function(i))) // Potentially unsafe

addr := (uintptr)(unsafe.Pointer(p))
fooi := (*Foo)(unsafe.Pointer(addr + i)) // Unsafe, even if you can
promise p points to the heap, and a strong reference to p exists, it can
still break in the future

Anything else is not guaranteed to be safe by Go. It is currently unsafe
unless you can prove that the object is on the heap and an anchored pointer
exists to it. And it won't be safe at all in the future if a moving GC is
introduced.

On Wednesday, July 15, 2015 at 12:12:12 PM UTC-5, jon.f...@gmail.com
wrote:

On Thursday, 9 July 2015 18:21:09 UTC+10, Alexander Neumann wrote:

So if anybody has an idea on how to debug this, I would be happy to hear
about it.

On Thursday, July 9, 2015 at 1:43:40 AM UTC+2, brainman wrote:

Welcome back :-)

Alex