FAQ
*Hi,

Eventually, I’d like to write shared libraries in go, in particular I want
my go code callable from the android dalvik virtual machine. So, after a
long and intense staring contest with the tools and the go runtime, I have
a proof of concept running under linux/x64. I know that your contribution
guide mentions that I should write my design before coding, but in this
case I chose to do the experiments myself first to get a feel of the go
code and to figure out if it was even possible for me to implement.

With the two CLs applied, building the runtime with PIC support is done with

GO_FLAGS=-lib ./make.bash

after which you can run the attached test case like this:

./build.sh && LD_LIBRARY_PATH=. ./main

which should give the following output:

Main, calling cgo
From test, calling go
Go callback!
C main, calling Go_callback
Go callback!
C main exiting

The test case demonstrates a C main program loading a go shared library
(libgo.so), which in turn loads a C shared library (libtest.so). The test
case calling sequence is as follows:

1. libgo.so DT_INIT is run by the dynamic linker. Go is initialized and
main.main() is run. In this case, libtest.so is called from main.main
through cgo.
2. main.main() returns and instead of exiting, control is relinquished back
to the dynamic linker which in turn calls the C main().
3. C main() calls back into libgo.so through cgo and exits.

The tool changes (http://codereview.appspot.com/6775101) adds the new
“-lib” flag to the go tool and the -U flags to 6l, 6g and 6c. With -U the
6g and 6c tools outputs code compatible with position independent code. 6l
in turn outputs the necessary relocations and uses RIP-relative addressing
in the resulting machine code. It also sets up DT_INIT to initialize the
library and it marks the resulting elf file as a shared library instead of
a regular executable. The resulting file is an elf shared library (ET_DYN)
and can be used both as a executable and as a shared library. When run as
an executable, there is the added benefit of activating ASLR since the file
is recognized as position independent.

The runtime changes (http://codereview.appspot.com/6822078) adds runtime
support for go shared libraries. In particular:

1. Assembler changes to allow 6l to output RIP-relative addressing on x64.
2. Scheduler changes to allow the main go routine to exit cleanly and
relinquish control back to the main program (proc.c).
3. Changes to the stack unwinding code to compensate for the runtime
relocations (symtab.c).
4. Changes the VDSO loader to use /proc/maps instead of the ancillary
vector which is not available from the library init (vdso_linux_amd64.c and
runtime.c)

I don’t expect the changes to go in as is, but I would very much welcome
reviews, comments etc., to allow for go shared library support, eventually
on the other platforms and archs too.

  - elias*

--

Search Discussions

  • Bryanturley at Nov 3, 2012 at 4:36 pm
    What happens when you run two go based .so files?
    If they are both making threads thinking they are the only go runtime it
    could lead to some interesting chaos.

    Also this probably should have been posted on golang-dev I think.

    --
  • Elias Naur at Nov 3, 2012 at 4:46 pm

    On Saturday, November 3, 2012 5:36:54 PM UTC+1, bryanturley wrote:
    What happens when you run two go based .so files?
    If they are both making threads thinking they are the only go runtime it
    could lead to some interesting chaos.

    Also this probably should have been posted on golang-dev I think.
    I've deleted the original post)I've and replied to this in the cross-post
    in golang-dev. Sorry for the inconvenience.

      - elias

    --
  • Elias Naur at Nov 3, 2012 at 4:48 pm

    On Saturday, November 3, 2012 5:46:53 PM UTC+1, Elias Naur wrote:

    On Saturday, November 3, 2012 5:36:54 PM UTC+1, bryanturley wrote:

    What happens when you run two go based .so files?
    If they are both making threads thinking they are the only go runtime it
    could lead to some interesting chaos.

    Also this probably should have been posted on golang-dev I think.
    I've deleted the original post)I've and replied to this in the cross-post
    in golang-dev. Sorry for the inconvenience.

    - elias
    That's what I thought, at least. Apparently I need to go through moderation
    before my replies appear.

      - elias

    --
  • Bryanturley at Nov 3, 2012 at 4:49 pm

    Also this probably should have been posted on golang-dev I think.
    I've deleted the original post)I've and replied to this in the cross-post
    in golang-dev. Sorry for the inconvenience.

    - elias
    Meh, don't think it was that big of a deal, or even a deal at all really.

    --
  • Elias Naur at Nov 3, 2012 at 5:01 pm

    On Saturday, November 3, 2012 5:36:54 PM UTC+1, bryanturley wrote:
    What happens when you run two go based .so files?
    If they are both making threads thinking they are the only go runtime it
    could lead to some interesting chaos.
    I haven't tried to address that issue, but I suspect that with proper
    hiding of the shared library symbols, two go shared libraries could happily
    coexist, with two seperate runtimes, blissfully unaware of each other. Not
    the most efficient solution, but certainly simpler that trying to let two
    go libs share a single, global go runtime.

      - elias

    --
  • Minux at Nov 3, 2012 at 5:49 pm

    On Sun, Nov 4, 2012 at 1:01 AM, Elias Naur wrote:

    I haven't tried to address that issue, but I suspect that with proper
    hiding of the shared library symbols, two go shared libraries could happily
    coexist, with two seperate runtimes, blissfully unaware of each other. Not
    the most efficient solution, but certainly simpler that trying to let two
    go libs share a single, global go runtime.
    consider sole reference to memory allocated from one runtime passed to
    Go code running under the other runtime, the first runtime will consider
    the memory garbage, but it's used in fact.

    This is part of the problem why Go doesn't produce dynamic library
    now.
    Another complexity is, Go never guarantee stable ABI, with static linking,
    this doesn't matter; but with dynamic linking, we will need some kind of
    versioning system to prevent from loading ABI-incompatible libraries.
    (yes, i know ELF has symbol versioning and soname mechanism, but
    what about Mach-O and PE/COFF?)

    --
  • Elias Naur at Nov 3, 2012 at 5:55 pm
    On Sat, Nov 3, 2012 at 6:49 PM, minux wrote:
    On Sun, Nov 4, 2012 at 1:01 AM, Elias Naur wrote:

    I haven't tried to address that issue, but I suspect that with proper
    hiding of the shared library symbols, two go shared libraries could happily
    coexist, with two seperate runtimes, blissfully unaware of each other. Not
    the most efficient solution, but certainly simpler that trying to let two
    go libs share a single, global go runtime.
    consider sole reference to memory allocated from one runtime passed to
    Go code running under the other runtime, the first runtime will consider
    the memory garbage, but it's used in fact.
    How is that different from the problem of passing references to C code
    through cgo? AFAIK, the proper way to do that is to hold a reference to the
    data on the go side while it is accessed from C.

    This is part of the problem why Go doesn't produce dynamic library
    now.
    Another complexity is, Go never guarantee stable ABI, with static linking,
    this doesn't matter; but with dynamic linking, we will need some kind of
    versioning system to prevent from loading ABI-incompatible libraries.
    (yes, i know ELF has symbol versioning and soname mechanism, but
    what about Mach-O and PE/COFF?)
    Do you mean in the one-global-runtime case? If two go libraries are
    completely seperate with seperate runtimes, schedulers etv. you shouldn't
    need ABI stability other than what the C interface gives you.

      - elias

    --
  • Minux at Nov 3, 2012 at 6:07 pm

    On Sun, Nov 4, 2012 at 1:54 AM, Elias Naur wrote:
    On Sat, Nov 3, 2012 at 6:49 PM, minux wrote:
    On Sun, Nov 4, 2012 at 1:01 AM, Elias Naur wrote:

    I haven't tried to address that issue, but I suspect that with proper
    hiding of the shared library symbols, two go shared libraries could happily
    coexist, with two seperate runtimes, blissfully unaware of each other. Not
    the most efficient solution, but certainly simpler that trying to let two
    go libs share a single, global go runtime.
    consider sole reference to memory allocated from one runtime passed to
    Go code running under the other runtime, the first runtime will consider
    the memory garbage, but it's used in fact.

    How is that different from the problem of passing references to C code
    through cgo? AFAIK, the proper way to do that is to hold a reference to the
    data on the go side while it is accessed from C.
    This problem is vastly different. Passing pointer to cgo is crossing
    language borders, but
    passing Go memory to Go (under another runtime) shouldn't be regarded as
    crossing
    language borders, and should be fully transparent to the user.

    Consider the the most common use case of dynamic loading Go code: dynamic
    plugin.
    do you really think that the user must make sure the main application
    retain every
    pointer passed to the plugin? Then we've returned to the old way of manual
    memory
    management, and in sometimes, this won't even be possible.
    This is part of the problem why Go doesn't produce dynamic library
    now.
    Another complexity is, Go never guarantee stable ABI, with static linking,
    this doesn't matter; but with dynamic linking, we will need some kind of
    versioning system to prevent from loading ABI-incompatible libraries.
    (yes, i know ELF has symbol versioning and soname mechanism, but
    what about Mach-O and PE/COFF?)
    Do you mean in the one-global-runtime case? If two go libraries are
    completely seperate with seperate runtimes, schedulers etv. you shouldn't
    need ABI stability other than what the C interface gives you.
    Yes. IMHO, one runtime is a must (of course, you can embed one runtime into
    each
    dynamic library, but only one of them can be in charge at any time).

    however, embedding one runtime into each dynamic Go library defeated the
    space
    saving for dynamic loading.

    --
  • Elias Naur at Nov 3, 2012 at 6:21 pm
    On Sat, Nov 3, 2012 at 7:07 PM, minux wrote:
    On Sun, Nov 4, 2012 at 1:54 AM, Elias Naur wrote:
    On Sat, Nov 3, 2012 at 6:49 PM, minux wrote:
    On Sun, Nov 4, 2012 at 1:01 AM, Elias Naur wrote:

    I haven't tried to address that issue, but I suspect that with proper
    hiding of the shared library symbols, two go shared libraries could happily
    coexist, with two seperate runtimes, blissfully unaware of each other. Not
    the most efficient solution, but certainly simpler that trying to let two
    go libs share a single, global go runtime.
    consider sole reference to memory allocated from one runtime passed to
    Go code running under the other runtime, the first runtime will consider
    the memory garbage, but it's used in fact.

    How is that different from the problem of passing references to C code
    through cgo? AFAIK, the proper way to do that is to hold a reference to the
    data on the go side while it is accessed from C.
    This problem is vastly different. Passing pointer to cgo is crossing
    language borders, but
    passing Go memory to Go (under another runtime) shouldn't be regarded as
    crossing
    language borders, and should be fully transparent to the user.

    Consider the the most common use case of dynamic loading Go code: dynamic
    plugin.
    do you really think that the user must make sure the main application
    retain every
    pointer passed to the plugin? Then we've returned to the old way of manual
    memory
    management, and in sometimes, this won't even be possible.
    This is part of the problem why Go doesn't produce dynamic library
    now.
    Another complexity is, Go never guarantee stable ABI, with static
    linking,
    this doesn't matter; but with dynamic linking, we will need some kind of
    versioning system to prevent from loading ABI-incompatible libraries.
    (yes, i know ELF has symbol versioning and soname mechanism, but
    what about Mach-O and PE/COFF?)
    Do you mean in the one-global-runtime case? If two go libraries are
    completely seperate with seperate runtimes, schedulers etv. you shouldn't
    need ABI stability other than what the C interface gives you.
    Yes. IMHO, one runtime is a must (of course, you can embed one runtime
    into each
    dynamic library, but only one of them can be in charge at any time).

    however, embedding one runtime into each dynamic Go library defeated the
    space
    saving for dynamic loading.
    This is the reason we're discussing this at all :) I'm perfectly happy with
    statically linked stand-alone executables for my go programs, and not
    particularly interested in the space savings in disk space or runtime
    memory. My focus is the cases where a shared library is an absolute
    requirement. A great example is the android dalvik java runtime which
    _requires_ external native code to be in the form of a shared library. But
    anytime you'd like to write go plugins of any sort you'll very likely need
    dynamic library support in some form.

    How multiple plugins written in go coexist is definitely an issue, but I'm
    willing to go with "unsupported" in the short run, "works, but inefficient
    in disk and memory space" in the longer run and some kind of "runtime
    sharing of code/runtime/memory" for the longest run.

    But for now, I'd just like to get the basics working - position independent
    code, runtime changes, compiler flags etc which is exactly what my CLs
    address. As far as I see, you can't avoid that work, however you want to
    support go shared libraries in the future.

      - elias

    --
  • Bryanturley at Nov 4, 2012 at 4:49 am

    This is the reason we're discussing this at all :) I'm perfectly happy
    with statically linked stand-alone executables for my go programs, and not
    particularly interested in the space savings in disk space or runtime
    memory. My focus is the cases where a shared library is an absolute
    requirement. A great example is the android dalvik java runtime which
    _requires_ external native code to be in the form of a shared library. But
    anytime you'd like to write go plugins of any sort you'll very likely need
    dynamic library support in some form.
    It is incredibly annoying that the android team wants everyone to use java
    only, then lets you load just about any random .so inside your java program.
    I want to use go on android as well, and I know I have seen other threads
    to that end.
    I wonder how many other languages aren't represented on android because of
    this decision.
    The answer seems simple, FIX ANDROID, but that doesn't seem likely.
    Sorry, I may have some pent up anger at the android team.... ;)

    How multiple plugins written in go coexist is definitely an issue, but I'm
    willing to go with "unsupported" in the short run, "works, but inefficient
    in disk and memory space" in the longer run and some kind of "runtime
    sharing of code/runtime/memory" for the longest run.

    But for now, I'd just like to get the basics working - position
    independent code, runtime changes, compiler flags etc which is exactly what
    my CLs address. As far as I see, you can't avoid that work, however you
    want to support go shared libraries in the future.

    - elias
      go build -dirty-android-hack
      I would use it.
      I should prob dig into the go code deeper myself see if I can help.

    --
  • Elias Naur at Nov 4, 2012 at 9:14 am

    On Sun, Nov 4, 2012 at 5:49 AM, bryanturley wrote:

    This is the reason we're discussing this at all :) I'm perfectly happy
    with statically linked stand-alone executables for my go programs, and not
    particularly interested in the space savings in disk space or runtime
    memory. My focus is the cases where a shared library is an absolute
    requirement. A great example is the android dalvik java runtime which
    _requires_ external native code to be in the form of a shared library. But
    anytime you'd like to write go plugins of any sort you'll very likely need
    dynamic library support in some form.
    It is incredibly annoying that the android team wants everyone to use java
    only, then lets you load just about any random .so inside your java program.
    I want to use go on android as well, and I know I have seen other threads
    to that end.
    I wonder how many other languages aren't represented on android because of
    this decision.
    The answer seems simple, FIX ANDROID, but that doesn't seem likely.
    Sorry, I may have some pent up anger at the android team.... ;)

    How multiple plugins written in go coexist is definitely an issue, but I'm
    willing to go with "unsupported" in the short run, "works, but inefficient
    in disk and memory space" in the longer run and some kind of "runtime
    sharing of code/runtime/memory" for the longest run.

    But for now, I'd just like to get the basics working - position
    independent code, runtime changes, compiler flags etc which is exactly what
    my CLs address. As far as I see, you can't avoid that work, however you
    want to support go shared libraries in the future.

    - elias
    go build -dirty-android-hack
    I would use it.
    I should prob dig into the go code deeper myself see if I can help.

    --


    Android is an especially annoying use case, since go already runs on
    android as stand alone executables, they're just no distributable as
    standard .apk files through Play and the like. Only way to do that is to
    run an embedded go program from java and communicate with it with some kind
    of IPC.

    However, there are still many other cases where you have a main program
    extensible with plugins, and that is where shared libraries is the most
    common way to do that. Another useful case is where you have a "friendly"
    program that is either written entirely in Go or simply wants Go as a
    "scripting language" not unlike what Python is today. In that case, I could
    imagine shared library support would be be useful.

      - elias

    --
  • Bryanturley at Nov 4, 2012 at 3:09 pm

    Android is an especially annoying use case, since go already runs on
    android as stand alone executables, they're just no distributable as
    standard .apk files through Play and the like. Only way to do that is to
    run an embedded go program from java and communicate with it with some kind
    of IPC.

    However, there are still many other cases where you have a main program
    extensible with plugins, and that is where shared libraries is the most
    common way to do that. Another useful case is where you have a "friendly"
    program that is either written entirely in Go or simply wants Go as a
    "scripting language" not unlike what Python is today. In that case, I could
    imagine shared library support would be be useful.

    - elias
    Yes I am aware, I have written pure c programs for android just not inside
    of a .apk. I have written some small toys with the NativeActivity wrapper
    garbage as well.
    What I can't understand is why won't they just allow fully native code from
    a .apk, they could keep their all programs have a different uid scheme and
    permissions xml data and what not and just run a binary instead of a some
    java that loads a native .so.
    It is a baffling.

    --
  • 小菜 at Jan 11, 2015 at 1:52 pm
    Don't understand!


    在 2012年11月4日星期日 UTC+8上午12:36:54,Bryan Turley写道:
    What happens when you run two go based .so files?
    If they are both making threads thinking they are the only go runtime it
    could lead to some interesting chaos.

    Also this probably should have been posted on golang-dev I think.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Bryan Turley at Jan 11, 2015 at 5:45 pm
    There was a message before that one. Elias Naur was showing his go on android work.
    This later evolved into the current go on android stuff.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Rémy Oudompheng at Nov 3, 2012 at 4:47 pm

    On 2012/11/3 Elias Naur wrote:
    Hi,

    Eventually, I’d like to write shared libraries in go, in particular I want
    my go code callable from the android dalvik virtual machine. So, after a
    long and intense staring contest with the tools and the go runtime, I have a
    proof of concept running under linux/x64. I know that your contribution
    guide mentions that I should write my design before coding, but in this case
    I chose to do the experiments myself first to get a feel of the go code and
    to figure out if it was even possible for me to implement.
    At first I thought the change was about turning Go packages into
    shared libraries but that's not what it is about, it's about embedding
    a Go program/library into an external program.

    Rémy.

    --
  • Elias Naur at Nov 3, 2012 at 4:49 pm

    On Saturday, November 3, 2012 5:47:14 PM UTC+1, Rémy Oudompheng wrote:
    On 2012/11/3 Elias Naur <elias...@gmail.com <javascript:>> wrote:
    Hi,

    Eventually, I’d like to write shared libraries in go, in particular I want
    my go code callable from the android dalvik virtual machine. So, after a
    long and intense staring contest with the tools and the go runtime, I have a
    proof of concept running under linux/x64. I know that your contribution
    guide mentions that I should write my design before coding, but in this case
    I chose to do the experiments myself first to get a feel of the go code and
    to figure out if it was even possible for me to implement.
    At first I thought the change was about turning Go packages into
    shared libraries but that's not what it is about, it's about embedding
    a Go program/library into an external program.

    Rémy.
    Exactly.

      - elias

    --
  • Minux at Nov 3, 2012 at 4:50 pm

    On Sun, Nov 4, 2012 at 12:27 AM, Elias Naur wrote:

    *Eventually, I’d like to write shared libraries in go, in particular I
    want my go code callable from the android dalvik virtual machine. So, after
    a long and intense staring contest with the tools and the go runtime, I
    have a proof of concept running under linux/x64. I know that your
    contribution guide mentions that I should write my design before coding,
    but in this case I chose to do the experiments myself first to get a feel
    of the go code and to figure out if it was even possible for me to
    implement.*
    Please take a look at http://code.google.com/p/go/issues/detail?id=4069.
    I think that's the route to go for producing dynamic libraries.

    btw, according to contribution guide, this thread does belong to
    golang-nuts.

    --
  • Elias Naur at Nov 3, 2012 at 4:55 pm

    On Saturday, November 3, 2012 5:50:55 PM UTC+1, minux wrote:

    On Sun, Nov 4, 2012 at 12:27 AM, Elias Naur <elias...@gmail.com<javascript:>
    wrote:
    *Eventually, I’d like to write shared libraries in go, in particular I
    want my go code callable from the android dalvik virtual machine. So, after
    a long and intense staring contest with the tools and the go runtime, I
    have a proof of concept running under linux/x64. I know that your
    contribution guide mentions that I should write my design before coding,
    but in this case I chose to do the experiments myself first to get a feel
    of the go code and to figure out if it was even possible for me to
    implement.*
    Please take a look at http://code.google.com/p/go/issues/detail?id=4069.
    I think that's the route to go for producing dynamic libraries.
    I've seen that, but I believe that my work is a subset of that solution,
    since you still need PIC and runtime support to make .o files useful for
    gcc linking into shared libraries (.so).

    btw, according to contribution guide, this thread does belong to
    golang-nuts.
    Oh my, what a mess :) golang-nuts is best for me anyway, because I
    currently don't have posting ability in golang-devl. For reference, the
    golang-dev thread is at:

    https://groups.google.com/d/topic/golang-dev/M8w4TolblDg/discussion

      - elias

    --
  • Patrick Mylund Nielsen at Nov 3, 2012 at 5:00 pm
    Now everybody knows, at least :)

    On Sat, Nov 3, 2012 at 5:55 PM, Elias Naur wrote:


    On Saturday, November 3, 2012 5:50:55 PM UTC+1, minux wrote:

    On Sun, Nov 4, 2012 at 12:27 AM, Elias Naur wrote:

    *Eventually, I’d like to write shared libraries in go, in particular I
    want my go code callable from the android dalvik virtual machine. So, after
    a long and intense staring contest with the tools and the go runtime, I
    have a proof of concept running under linux/x64. I know that your
    contribution guide mentions that I should write my design before coding,
    but in this case I chose to do the experiments myself first to get a feel
    of the go code and to figure out if it was even possible for me to
    implement.*
    Please take a look at http://code.google.com/p/go/**issues/detail?id=4069<http://code.google.com/p/go/issues/detail?id=4069>
    .
    I think that's the route to go for producing dynamic libraries.
    I've seen that, but I believe that my work is a subset of that solution,
    since you still need PIC and runtime support to make .o files useful for
    gcc linking into shared libraries (.so).

    btw, according to contribution guide, this thread does belong to
    golang-nuts.
    Oh my, what a mess :) golang-nuts is best for me anyway, because I
    currently don't have posting ability in golang-devl. For reference, the
    golang-dev thread is at:

    https://groups.google.com/d/topic/golang-dev/M8w4TolblDg/discussion

    - elias

    --

    --
  • Bryanturley at Nov 4, 2012 at 4:30 am

    On Saturday, November 3, 2012 11:50:55 AM UTC-5, minux wrote:

    On Sun, Nov 4, 2012 at 12:27 AM, Elias Naur <elias...@gmail.com<javascript:>
    wrote:
    *Eventually, I’d like to write shared libraries in go, in particular I
    want my go code callable from the android dalvik virtual machine. So, after
    a long and intense staring contest with the tools and the go runtime, I
    have a proof of concept running under linux/x64. I know that your
    contribution guide mentions that I should write my design before coding,
    but in this case I chose to do the experiments myself first to get a feel
    of the go code and to figure out if it was even possible for me to
    implement.*
    Please take a look at http://code.google.com/p/go/issues/detail?id=4069.
    I think that's the route to go for producing dynamic libraries.

    btw, according to contribution guide, this thread does belong to
    golang-nuts.
    Ooops. My bad ;)

    --
  • Greg Ward at Nov 7, 2012 at 3:12 am

    On 03 November 2012, Elias Naur said:
    Eventually, I’d like to write shared libraries in go, in particular I want
    my go code callable from the android dalvik virtual machine. So, after a
    long and intense staring contest with the tools and the go runtime, I have
    a proof of concept running under linux/x64. Nifty!
    With the two CLs applied, building the runtime with PIC support is done with

    GO_FLAGS=-lib ./make.bash
    Bikeshedding, since I'm completely ignorant of all the low-level
    nitty-gritty details required to make this work ... but I do know how
    to use GCC to build shared libraries from C, and the relevant option
    there is -shared. So why not -shared instead of -lib?

            Greg

    --
  • Elias Naur at Nov 7, 2012 at 10:14 am

    On Wed, Nov 7, 2012 at 4:12 AM, Greg Ward wrote:

    With the two CLs applied, building the runtime with PIC support is done with
    GO_FLAGS=-lib ./make.bash
    Bikeshedding, since I'm completely ignorant of all the low-level
    nitty-gritty details required to make this work ... but I do know how
    to use GCC to build shared libraries from C, and the relevant option
    there is -shared. So why not -shared instead of -lib?

    Greg
    Sensible bikeshedding - I've updated the CLs to use -shared instead of
    -lib. I've also sync'ed to tip and verified all.bash still passes.

      - elias

    --
  • Bryanturley at Nov 7, 2012 at 5:00 pm
    Have you tried it on arm yet?

    --
  • Elias Naur at Nov 7, 2012 at 6:19 pm

    On Wed, Nov 7, 2012 at 6:00 PM, bryanturley wrote:

    Have you tried it on arm yet?

    Not yet. Obviously ARM support is required for go Android libraries to
    work, but I started with x64 to get the generic bits done and because of
    familarity of the architecture and for convenience (I don't have access to
    an ARM based machine other than my phone).

    At the moment, I'm holding my breath for some indication on whether my CLs
    are the right direction to go before I spend more time on the 5* arch
    specific parts.

      - elias

    --
  • Elias Naur at Dec 11, 2012 at 3:27 pm
    Hi,

    It's been a while, so I hg sync'ed and made a few bug fixes to my changes
    to it work with the current tip again. hg change <CL Id> didn't update the
    files for me, so I created a new CL at 6926049. Any pointers as to how to
    update the CL patch set would be much appreciated. Was it because I didn't
    change the description? Changes:

    *

    - a missing -lib to -shared conversion in cmd/go/testflags.go

    - Removed unused typedef from cmd/6l/l.h

    - moved rel_ro assignment into -U flag conditional in cmd/ld/data.c

    - synced to tip and fixed, all tests again pass with or without
    GO_FLAGS=-shared


    *
    Additionally, I updated the test case to not hardcode paths. Download it,
    unzip it and run it with:

    GOPATH=<absolute path to test dir> GOROOT=<path to go root built with
    GO_FLAGS=-shared> ./build.sh && LD_LIBRARY_PATH=. ./main*
    *

    And lastly, I copied the original post below, since it was deleted by me
    because of confusion as to whether the post belonged in golang-dev.

    - elias
    *
    *
    *
    *
    *Original post:*
    *
    *
    *Eventually, I’d like to write shared libraries in go, in particular I want
    my go code callable from the android dalvik virtual* machine. So, after a
    long and intense staring contest with the tools and the go runtime, I have
    a proof of concept running under linux/x64. I know that your contribution
    guide mentions that I should write my design before coding, but in this
    case I chose to do the experiments myself first to get a feel of the go
    code and to figure out if it was even possible for me to implement.

    With the two CLs applied, building the runtime with PIC support is done with

    GO_FLAGS=-shared ./all.bash

    after which you can run the attached test case like this:

    GOPATH=<absolute path to test dir> GOROOT=<path to go root built with
    GO_FLAGS=-shared> ./build.sh && LD_LIBRARY_PATH=. ./main

    which should give the following output:

    Main, calling cgo From test, calling go
    Go callback!
    C main, calling Go_callback
    Go callback!
    C main exiting

    The test case demonstrates a C main program loading a go shared library
    (libgo.so), which in turn loads a C shared library (libtest.so). The test
    case calling sequence is as follows:

    1. libgo.so DT_INIT is run by the dynamic linker. Go is initialized and
    main.main() is run. In this case, libtest.so is called from main.main
    through cgo.
    2. main.main() returns and instead of exiting, control is relinquished back
    to the dynamic linker which in turn calls the C main().
    3. C main() calls back into libgo.so through cgo and exits.

    The tool changes (http://codereview.appspot.com/6775101) adds the new
    “-lib” flag to the go tool and the -U flags to 6l, 6g and 6c. With -U the
    6g and 6c tools outputs code compatible with position independent code. 6l
    in turn outputs the necessary relocations and uses RIP-relative addressing
    in the resulting machine code. It also sets up DT_INIT to initialize the
    library and it marks the resulting elf file as a shared library instead of
    a regular executable. The resulting file is an elf shared library (ET_DYN)
    and can be used both as a executable and as a shared library. When run as
    an executable, there is the added benefit of activating ASLR since the file
    is recognized as position independent.

    The runtime changes (http://codereview.appspot.com/6822078) adds runtime
    support for go shared libraries. In particular:

    1. Assembler changes to allow 6l to output RIP-relative addressing on x64.
    2. Scheduler changes to allow the main go routine to exit cleanly and
    relinquish control back to the main program (proc.c).
    3. Changes to the stack unwinding code to compensate for the runtime
    relocations (symtab.c).
    4. Changes the VDSO loader to use /proc/maps instead of the ancillary
    vector which is not available from the library init (vdso_linux_amd64.c and
    runtime.c)

    I don’t expect the changes to go in as is, but I would very much welcome
    reviews, comments etc., to allow for go shared library support, eventually
    on the other platforms and archs too.

      - elias

    *
    *

    --
  • Ian Lance Taylor at Dec 11, 2012 at 5:13 pm

    On Tue, Dec 11, 2012 at 7:27 AM, Elias Naur wrote:
    It's been a while, so I hg sync'ed and made a few bug fixes to my changes to
    it work with the current tip again. hg change <CL Id> didn't update the
    files for me, so I created a new CL at 6926049. Any pointers as to how to
    update the CL patch set would be much appreciated. Was it because I didn't
    change the description?
    You shouldn't need to update the description. You should be able to
    just change the list of files. "hg pending" should show the CLs in
    your working directory, and also list the files that are not in any
    CL. Once you update the CL in your working directory, "hg upload
    NNNNNN" will upload it to the codereview site.

    Ian

    --
  • Elias Naur at Dec 11, 2012 at 5:38 pm

    On Tue, Dec 11, 2012 at 6:05 PM, Ian Lance Taylor wrote:
    On Tue, Dec 11, 2012 at 7:27 AM, Elias Naur wrote:

    It's been a while, so I hg sync'ed and made a few bug fixes to my
    changes to
    it work with the current tip again. hg change <CL Id> didn't update the
    files for me, so I created a new CL at 6926049. Any pointers as to how to
    update the CL patch set would be much appreciated. Was it because I didn't
    change the description?
    You shouldn't need to update the description. You should be able to
    just change the list of files. "hg pending" should show the CLs in
    your working directory, and also list the files that are not in any
    CL. Once you update the CL in your working directory, "hg upload
    NNNNNN" will upload it to the codereview site.

    Ian
    Hi Ian,

    Thank you. Yes, Russ also pointed me to "hg upload" which was the command I
    needed. I can see that the contribution guide lists "hg mail" which
    apparently does the same plus mails the CL to reviewers, but I didn't want
    to bother reviewers. If there's not a list of commands in the mercurial
    extension elsewhere, maybe hg upload and the rest of the mercurial
    extension commands should also be listed in the guide?

      - elias

    --
  • Russ Cox at Dec 11, 2012 at 5:43 pm
    'hg help codereview' will show the codereview commands. I will add a
    note about that to the web page.

    --
  • Elias Naur at Dec 18, 2012 at 7:20 pm
    I've updated the changesets to the latest revision and verifed everything
    works. Minor change:

      - The tools CL (6926049) fixed a small symbol generation bug, which was
    needed for shared libraries to work. A slightly different fix for the same
    bug was recently fixed by minux, so I've integrated that in 6926049.

    Russ and everyone else: Will it make any difference to "hg mail" the
    changesets or will it simply annoy?

      - elias

    --
  • Mike Rosset at Dec 19, 2012 at 6:19 pm
    you should hg mail. since that will mail to golang-dev.
    On Tue, Dec 18, 2012 at 11:20 AM, Elias Naur wrote:
    I've updated the changesets to the latest revision and verifed everything
    works. Minor change:

    - The tools CL (6926049) fixed a small symbol generation bug, which was
    needed for shared libraries to work. A slightly different fix for the same
    bug was recently fixed by minux, so I've integrated that in 6926049.

    Russ and everyone else: Will it make any difference to "hg mail" the
    changesets or will it simply annoy?

    - elias

    --
    --
  • Mike Rosset at Dec 24, 2012 at 8:39 am
    Elias did you mail this patch off? I cant seem to add comments to it.
    I was wondering what the status was.
    On Wed, Dec 19, 2012 at 10:19 AM, Mike Rosset wrote:
    you should hg mail. since that will mail to golang-dev.
    On Tue, Dec 18, 2012 at 11:20 AM, Elias Naur wrote:
    I've updated the changesets to the latest revision and verifed everything
    works. Minor change:

    - The tools CL (6926049) fixed a small symbol generation bug, which was
    needed for shared libraries to work. A slightly different fix for the same
    bug was recently fixed by minux, so I've integrated that in 6926049.

    Russ and everyone else: Will it make any difference to "hg mail" the
    changesets or will it simply annoy?

    - elias

    --
    --
  • Minux at Dec 24, 2012 at 8:48 am

    On Monday, December 24, 2012, Mike Rosset wrote:

    Elias did you mail this patch off? I cant seem to add comments to it.
    you can add and mail comments even the CL is not mailed out.

    note to the cl author: if you don't want to automatically mail out
    review request for the CL
    as a side effect of replying to comments, please reply from your
    email client, not from reitveld.

    --
  • Elias Naur at Dec 24, 2012 at 5:37 pm
    Hi Mike,

    Yes, I've now hg mail'ed the CLs

      - elias
    On Monday, December 24, 2012 9:39:49 AM UTC+1, Mike Rosset wrote:

    Elias did you mail this patch off? I cant seem to add comments to it.
    I was wondering what the status was.
    On Wed, Dec 19, 2012 at 10:19 AM, Mike Rosset wrote:
    you should hg mail. since that will mail to golang-dev.
    On Tue, Dec 18, 2012 at 11:20 AM, Elias Naur wrote:
    I've updated the changesets to the latest revision and verifed
    everything
    works. Minor change:

    - The tools CL (6926049) fixed a small symbol generation bug, which
    was
    needed for shared libraries to work. A slightly different fix for the
    same
    bug was recently fixed by minux, so I've integrated that in 6926049.

    Russ and everyone else: Will it make any difference to "hg mail" the
    changesets or will it simply annoy?

    - elias

    --
    --
  • Kevin Malachowski at Mar 26, 2013 at 2:49 pm
    Is there any recent progress on this? I can't seem to figure out how to get
    the -shared flag to work for 6l. It complains with the following:

    "entry not text: _rt0_amd64_linux_lib
    main.main: doasm: notfound from=78 to=8f (7) MOVQ
    $type.string+0(SB),(SP)
    (A whole bunch of other errors)
    "

    And all the program does is import "fmt" and tries to Println a simple
    string. go version reports

    "go version devel +08d20469cc20 Tue Mar 26 08:27:18 2013 +0100 linux/amd64"

    And I'm on an Ubuntu 12.04 virtual machine.

    In addition, when I try to use the -shared flag with `go build` it fails,
    complaining that the -shared flag was provided but not defined.
    On Monday, 24 December 2012 12:37:30 UTC-5, Elias Naur wrote:

    Hi Mike,

    Yes, I've now hg mail'ed the CLs

    - elias
    On Monday, December 24, 2012 9:39:49 AM UTC+1, Mike Rosset wrote:

    Elias did you mail this patch off? I cant seem to add comments to it.
    I was wondering what the status was.

    On Wed, Dec 19, 2012 at 10:19 AM, Mike Rosset <mike....@gmail.com>
    wrote:
    you should hg mail. since that will mail to golang-dev.

    On Tue, Dec 18, 2012 at 11:20 AM, Elias Naur <elias...@gmail.com>
    wrote:
    I've updated the changesets to the latest revision and verifed
    everything
    works. Minor change:

    - The tools CL (6926049) fixed a small symbol generation bug, which
    was
    needed for shared libraries to work. A slightly different fix for the
    same
    bug was recently fixed by minux, so I've integrated that in 6926049.

    Russ and everyone else: Will it make any difference to "hg mail" the
    changesets or will it simply annoy?

    - elias

    --
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Elias Naur at Mar 26, 2013 at 3:50 pm
    You'll need to pass the -largemodel flag to the Go and C compiler too, to
    avoid assembler instructions that can't use position independent
    addressing. However, the shared library work still need the runtime changes
    that were never merged. If nobody else beats me to it, I plan to update the
    work as soon as the current linker and cgo changes have settled down, some
    time after 1.1.

    The planning work for the semantic changes for the runtime is tracked here:

    https://code.google.com/p/go/issues/detail?id=4848<https://code.google.com/p/go/issues/detail?id=4848&q=elias&colspec=ID%20Status%20Stars%20Priority%20Owner%20Reporter%20Summary>

    Host linker support (issue 4069, complete for linux/darwin and i386/x64)
    and support for cgo callbacks from foreign threads (issue 4435, fixed) were
    recently added. With that, my best bet is splitting Go initialization to
    run separately, and only if Go is the main program, run main.main(). Other
    than that, there are various nitty gritty details to handle, as hinted in
    4848 (missing os.Args being one)

      - elias
    On Tuesday, March 26, 2013 3:49:18 PM UTC+1, Kevin Malachowski wrote:


    Is there any recent progress on this? I can't seem to figure out how to
    get the -shared flag to work for 6l. It complains with the following:

    "entry not text: _rt0_amd64_linux_lib
    main.main: doasm: notfound from=78 to=8f (7) MOVQ
    $type.string+0(SB),(SP)
    (A whole bunch of other errors)
    "

    And all the program does is import "fmt" and tries to Println a simple
    string. go version reports

    "go version devel +08d20469cc20 Tue Mar 26 08:27:18 2013 +0100 linux/amd64"

    And I'm on an Ubuntu 12.04 virtual machine.

    In addition, when I try to use the -shared flag with `go build` it fails,
    complaining that the -shared flag was provided but not defined.
    On Monday, 24 December 2012 12:37:30 UTC-5, Elias Naur wrote:

    Hi Mike,

    Yes, I've now hg mail'ed the CLs

    - elias
    On Monday, December 24, 2012 9:39:49 AM UTC+1, Mike Rosset wrote:

    Elias did you mail this patch off? I cant seem to add comments to it.
    I was wondering what the status was.

    On Wed, Dec 19, 2012 at 10:19 AM, Mike Rosset <mike....@gmail.com>
    wrote:
    you should hg mail. since that will mail to golang-dev.

    On Tue, Dec 18, 2012 at 11:20 AM, Elias Naur <elias...@gmail.com>
    wrote:
    I've updated the changesets to the latest revision and verifed
    everything
    works. Minor change:

    - The tools CL (6926049) fixed a small symbol generation bug, which
    was
    needed for shared libraries to work. A slightly different fix for the
    same
    bug was recently fixed by minux, so I've integrated that in 6926049.

    Russ and everyone else: Will it make any difference to "hg mail" the
    changesets or will it simply annoy?

    - elias

    --
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedNov 3, '12 at 4:27p
activeJan 11, '15 at 5:45p
posts36
users11
websitegolang.org

People

Translate

site design / logo © 2021 Grokbase