FAQ
The type system in GoMobile is still very limited, as it's currently not
possibly to work with slices of struct:
https://godoc.org/golang.org/x/mobile/cmd/gobind#hdr-Type_restrictions

In the meantime, developers are starting to release mobile apps with
GoMobile and they all have to deal with the same problem: the lack of
slices of struct. Each developer has been coming with its own solution.
Someone tried with wrappers
(https://github.com/scisci/go-mobile-collection), but the most convincing
solution so far seems to be Flatbuffers. A few developers have already used
it successfully.

Flatbuffers have official implementations in Go and Java. And there is
already an open source package for Swift:
https://github.com/mzaks/FlatBuffersSwift

I have talked to Robert Winslow, one of the guys behind Flatbuffers, if it
made sense to implement Flatbuffers for GoMobile and he seems to agree and
he is willing to help.

I am opening this thread, so that discussions can start.

--
You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Search Discussions

  • Joshua Foster at Jan 26, 2016 at 1:56 pm
    I would recommend considering Capn Proto. The solution is much more robust
    than Flatbuffers and has a better API.

    Joshua
    On Tuesday, January 26, 2016 at 8:29:41 AM UTC-5, Daniele Baroncelli wrote:

    The type system in GoMobile is still very limited, as it's currently not
    possibly to work with slices of struct:
    https://godoc.org/golang.org/x/mobile/cmd/gobind#hdr-Type_restrictions

    In the meantime, developers are starting to release mobile apps with
    GoMobile and they all have to deal with the same problem: the lack of
    slices of struct. Each developer has been coming with its own solution.
    Someone tried with wrappers (
    https://github.com/scisci/go-mobile-collection), but the most convincing
    solution so far seems to be Flatbuffers. A few developers have already used
    it successfully.

    Flatbuffers have official implementations in Go and Java. And there is
    already an open source package for Swift:
    https://github.com/mzaks/FlatBuffersSwift

    I have talked to Robert Winslow, one of the guys behind Flatbuffers, if it
    made sense to implement Flatbuffers for GoMobile and he seems to agree and
    he is willing to help.

    I am opening this thread, so that discussions can start.
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Matthew Sackman at Jan 26, 2016 at 3:03 pm

    On Tue, Jan 26, 2016 at 05:53:29AM -0800, Joshua Foster wrote:
    I would recommend considering Capn Proto. The solution is much more robust
    than Flatbuffers and has a better API.
    Interesting. At risk of this rapidly turning into bikeshedding, I've
    been using capnproto in Go for the last year. The Go bindings are fine,
    but the Javascript ones are currently broken, there seem to be some
    question marks about the Java ones etc. The only "official" one, I
    believe, is C++. A quick scan of the flatbuffers repo at
    https://github.com/google/flatbuffers/ suggests there are far more
    "official" language bindings for flatbuffers, and I'm much encouraged by
    the appearance of fuzz testing on
    https://google.github.io/flatbuffers/flatbuffers_support.html
    As I need good multiple language bindings, I'm currently considering
    moving from capnproto to flatbuffers, though I'm yet to use flatbuffers.

    I'm curious as to why you feel flatbuffers is less robust. The only
    comparison blogs I can find seem to be from a few years ago and don't
    tend to reflect using these things in anger for substantial projects.

    Matthew

    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Joshua Foster at Jan 26, 2016 at 3:24 pm
    When I tried getting a Flatbuffers version working, the unmarshall would
    never work. Flatbuffers also seemed to have more of a "create this, set
    this, create this, set this..." Capn Proto just worked.

    Joshua
    On Tuesday, January 26, 2016 at 10:03:32 AM UTC-5, Matthew Sackman wrote:
    On Tue, Jan 26, 2016 at 05:53:29AM -0800, Joshua Foster wrote:
    I would recommend considering Capn Proto. The solution is much more robust
    than Flatbuffers and has a better API.
    Interesting. At risk of this rapidly turning into bikeshedding, I've
    been using capnproto in Go for the last year. The Go bindings are fine,
    but the Javascript ones are currently broken, there seem to be some
    question marks about the Java ones etc. The only "official" one, I
    believe, is C++. A quick scan of the flatbuffers repo at
    https://github.com/google/flatbuffers/ suggests there are far more
    "official" language bindings for flatbuffers, and I'm much encouraged by
    the appearance of fuzz testing on
    https://google.github.io/flatbuffers/flatbuffers_support.html
    As I need good multiple language bindings, I'm currently considering
    moving from capnproto to flatbuffers, though I'm yet to use flatbuffers.

    I'm curious as to why you feel flatbuffers is less robust. The only
    comparison blogs I can find seem to be from a few years ago and don't
    tend to reflect using these things in anger for substantial projects.

    Matthew
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Hyang-Ah Hana Kim at Jan 26, 2016 at 11:07 pm
    Thanks for bringing up the issue.

    If the goal was to exchange data between two languages, I think use of
    flatbuffers or other equivalent technologies makes sense.
    Byte slices are currently supported, so it's possible to build your idea on
    top of gobind's byte slice support.

    But Gobind is more than just data passing; it offers cross-language object
    references, callbacks, etc.

    For example, Gobind allows Java-side to implement a Go interface, and pass
    the Java object to Go while holding the actual logic in Java.
    Passing a Go struct in Gobind also means that struct's method are also
    available to the other language.
    Maybe some methods assume the Go object maintain a certain state during its
    life time.
    I don't see how the use of Flatbuffers (or the equivalents) can help
    achieving these goal yet.

    Crawshaw and I have been going back and forth on how to support struct,
    slice of structs, etc.
    One option we considered is to promote the struct type to its pointer type.
    It seems doable but some may be unhappy about performance.
    Another proposal is to serialize the entire struct object and instantiate
    an object in Java or Objective-C based on the serialized data. It
    eliminates cross-language reference counting.
    But we are not sure about how to handle struct with non-trivial methods
    which may involve unexported fields to work.

    Running out of ideas, I am open to others' ideas...
    I believe, once this is settled, we can start looking into suitable
    encoding/decoding technologies wherever necessary.

    - Hana

    On Tue, Jan 26, 2016 at 6:49 AM, Daniele Baroncelli wrote:

    The type system in GoMobile is still very limited, as it's currently not
    possibly to work with slices of struct:
    https://godoc.org/golang.org/x/mobile/cmd/gobind#hdr-Type_restrictions

    In the meantime, developers are starting to release mobile apps with
    GoMobile and they all have to deal with the same problem: the lack of
    slices of struct. Each developer has been coming with its own solution.
    Someone tried with wrappers (
    https://github.com/scisci/go-mobile-collection), but the most convincing
    solution so far seems to be Flatbuffers. A few developers have already used
    it successfully.

    Flatbuffers have official implementations in Go and Java. And there is
    already an open source package for Swift:
    https://github.com/mzaks/FlatBuffersSwift

    I have talked to Robert Winslow, one of the guys behind Flatbuffers, if it
    made sense to implement Flatbuffers for GoMobile and he seems to agree and
    he is willing to help.

    I am opening this thread, so that discussions can start.

    --
    You received this message because you are subscribed to the Google Groups
    "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.


    --
    __

    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Daniele Baroncelli at Jan 27, 2016 at 4:29 pm
    Hi Hana,

    thanks for taking part in this discussion

    in my opinion, we need to take a step back and answer some basic questions:

    - what is the benefit of a GoMobile library?
    - what would developers use it for?
    - which kind of code would it make sense to write in Go and which one in
    Java/Swift?

    In this diagram I try to describe the typical use for GoMobile:
    goo.gl/UkVA9b

    In general I would say the Go part should be treated as a black box (only
    dealing with data and logic), and the threading and callbacks should be
    managed exclusively from the mobile language.

    For this reason, I think the Go library functions should just be called
    syncronously, providing input parameters and expecting just 1 output value
    (to be compatible with Java functions), without complex cross-language
    references. I think the binding should be kept as simple as possible.

    I would treat the Java/Swift -> Go interface as an API interface. The data
    is served by the Go library, and the Java/Swift just queries it, without
    too many interrelations.


    Daniele



    Il giorno mercoledì 27 gennaio 2016 00:07:41 UTC+1, Hyang-Ah Hana Kim ha
    scritto:
    Thanks for bringing up the issue.

    If the goal was to exchange data between two languages, I think use of
    flatbuffers or other equivalent technologies makes sense.
    Byte slices are currently supported, so it's possible to build your idea
    on top of gobind's byte slice support.

    But Gobind is more than just data passing; it offers cross-language object
    references, callbacks, etc.

    For example, Gobind allows Java-side to implement a Go interface, and pass
    the Java object to Go while holding the actual logic in Java.
    Passing a Go struct in Gobind also means that struct's method are also
    available to the other language.
    Maybe some methods assume the Go object maintain a certain state during
    its life time.
    I don't see how the use of Flatbuffers (or the equivalents) can help
    achieving these goal yet.

    Crawshaw and I have been going back and forth on how to support struct,
    slice of structs, etc.
    One option we considered is to promote the struct type to its pointer
    type. It seems doable but some may be unhappy about performance.
    Another proposal is to serialize the entire struct object and instantiate
    an object in Java or Objective-C based on the serialized data. It
    eliminates cross-language reference counting.
    But we are not sure about how to handle struct with non-trivial methods
    which may involve unexported fields to work.

    Running out of ideas, I am open to others' ideas...
    I believe, once this is settled, we can start looking into suitable
    encoding/decoding technologies wherever necessary.

    - Hana


    On Tue, Jan 26, 2016 at 6:49 AM, Daniele Baroncelli <
    daniele.b...@gmail.com <javascript:>> wrote:
    The type system in GoMobile is still very limited, as it's currently not
    possibly to work with slices of struct:
    https://godoc.org/golang.org/x/mobile/cmd/gobind#hdr-Type_restrictions

    In the meantime, developers are starting to release mobile apps with
    GoMobile and they all have to deal with the same problem: the lack of
    slices of struct. Each developer has been coming with its own solution.
    Someone tried with wrappers (
    https://github.com/scisci/go-mobile-collection), but the most convincing
    solution so far seems to be Flatbuffers. A few developers have already used
    it successfully.

    Flatbuffers have official implementations in Go and Java. And there is
    already an open source package for Swift:
    https://github.com/mzaks/FlatBuffersSwift

    I have talked to Robert Winslow, one of the guys behind Flatbuffers, if
    it made sense to implement Flatbuffers for GoMobile and he seems to agree
    and he is willing to help.

    I am opening this thread, so that discussions can start.

    --
    You received this message because you are subscribed to the Google Groups
    "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-dev+...@googlegroups.com <javascript:>.
    For more options, visit https://groups.google.com/d/optout.


    --
    __
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Daniele Baroncelli at Jan 27, 2016 at 4:38 pm
    Just one clarification:

    the "just 1 output value" should be in form of structured data
    hence a Java object matching the Flatbuffer schema, autogenerated following
    by the definition of the struct which is returned by the Go function

    Daniele

    Il giorno mercoledì 27 gennaio 2016 17:29:47 UTC+1, Daniele Baroncelli ha
    scritto:
    Hi Hana,

    thanks for taking part in this discussion

    in my opinion, we need to take a step back and answer some basic questions:

    - what is the benefit of a GoMobile library?
    - what would developers use it for?
    - which kind of code would it make sense to write in Go and which one in
    Java/Swift?

    In this diagram I try to describe the typical use for GoMobile:
    goo.gl/UkVA9b

    In general I would say the Go part should be treated as a black box (only
    dealing with data and logic), and the threading and callbacks should be
    managed exclusively from the mobile language.

    For this reason, I think the Go library functions should just be called
    syncronously, providing input parameters and expecting just 1 output value
    (to be compatible with Java functions), without complex cross-language
    references. I think the binding should be kept as simple as possible.

    I would treat the Java/Swift -> Go interface as an API interface. The data
    is served by the Go library, and the Java/Swift just queries it, without
    too many interrelations.


    Daniele



    Il giorno mercoledì 27 gennaio 2016 00:07:41 UTC+1, Hyang-Ah Hana Kim ha
    scritto:
    Thanks for bringing up the issue.

    If the goal was to exchange data between two languages, I think use of
    flatbuffers or other equivalent technologies makes sense.
    Byte slices are currently supported, so it's possible to build your idea
    on top of gobind's byte slice support.

    But Gobind is more than just data passing; it offers cross-language
    object references, callbacks, etc.

    For example, Gobind allows Java-side to implement a Go interface, and
    pass the Java object to Go while holding the actual logic in Java.
    Passing a Go struct in Gobind also means that struct's method are also
    available to the other language.
    Maybe some methods assume the Go object maintain a certain state during
    its life time.
    I don't see how the use of Flatbuffers (or the equivalents) can help
    achieving these goal yet.

    Crawshaw and I have been going back and forth on how to support struct,
    slice of structs, etc.
    One option we considered is to promote the struct type to its pointer
    type. It seems doable but some may be unhappy about performance.
    Another proposal is to serialize the entire struct object and instantiate
    an object in Java or Objective-C based on the serialized data. It
    eliminates cross-language reference counting.
    But we are not sure about how to handle struct with non-trivial methods
    which may involve unexported fields to work.

    Running out of ideas, I am open to others' ideas...
    I believe, once this is settled, we can start looking into suitable
    encoding/decoding technologies wherever necessary.

    - Hana


    On Tue, Jan 26, 2016 at 6:49 AM, Daniele Baroncelli <
    daniele.b...@gmail.com> wrote:
    The type system in GoMobile is still very limited, as it's currently not
    possibly to work with slices of struct:
    https://godoc.org/golang.org/x/mobile/cmd/gobind#hdr-Type_restrictions

    In the meantime, developers are starting to release mobile apps with
    GoMobile and they all have to deal with the same problem: the lack of
    slices of struct. Each developer has been coming with its own solution.
    Someone tried with wrappers (
    https://github.com/scisci/go-mobile-collection), but the most
    convincing solution so far seems to be Flatbuffers. A few developers have
    already used it successfully.

    Flatbuffers have official implementations in Go and Java. And there is
    already an open source package for Swift:
    https://github.com/mzaks/FlatBuffersSwift

    I have talked to Robert Winslow, one of the guys behind Flatbuffers, if
    it made sense to implement Flatbuffers for GoMobile and he seems to agree
    and he is willing to help.

    I am opening this thread, so that discussions can start.

    --
    You received this message because you are subscribed to the Google
    Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send
    an email to golang-dev+...@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.


    --
    __
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Daniel Skinner at Jan 27, 2016 at 5:05 pm
    what would developers use it for
    As Hana mentioned, gobind allows java to implement a go interface. This is
    something I found useful early on by declaring a Broadcaster interface in
    go and in java I implemented the Broadcaster.Stub to facilitate use of
    http://developer.android.com/reference/android/support/v4/content/LocalBroadcastManager.html
    as
    by company convention.

    It's not clear simply using flatbuffers accomplishes that type of goal.
    Right now, I'm still interested in the use of flatbuffers but this is
    because, as Hana mentioned, there's still a lot to work out with gobind.
    On Wed, Jan 27, 2016 at 10:29 AM Daniele Baroncelli wrote:

    Hi Hana,

    thanks for taking part in this discussion

    in my opinion, we need to take a step back and answer some basic questions:

    - what is the benefit of a GoMobile library?
    - what would developers use it for?
    - which kind of code would it make sense to write in Go and which one in
    Java/Swift?

    In this diagram I try to describe the typical use for GoMobile:
    goo.gl/UkVA9b

    In general I would say the Go part should be treated as a black box (only
    dealing with data and logic), and the threading and callbacks should be
    managed exclusively from the mobile language.

    For this reason, I think the Go library functions should just be called
    syncronously, providing input parameters and expecting just 1 output value
    (to be compatible with Java functions), without complex cross-language
    references. I think the binding should be kept as simple as possible.

    I would treat the Java/Swift -> Go interface as an API interface. The data
    is served by the Go library, and the Java/Swift just queries it, without
    too many interrelations.


    Daniele




    Il giorno mercoledì 27 gennaio 2016 00:07:41 UTC+1, Hyang-Ah Hana Kim ha
    scritto:
    Thanks for bringing up the issue.

    If the goal was to exchange data between two languages, I think use of
    flatbuffers or other equivalent technologies makes sense.
    Byte slices are currently supported, so it's possible to build your idea
    on top of gobind's byte slice support.

    But Gobind is more than just data passing; it offers cross-language
    object references, callbacks, etc.

    For example, Gobind allows Java-side to implement a Go interface, and
    pass the Java object to Go while holding the actual logic in Java.
    Passing a Go struct in Gobind also means that struct's method are also
    available to the other language.
    Maybe some methods assume the Go object maintain a certain state during
    its life time.
    I don't see how the use of Flatbuffers (or the equivalents) can help
    achieving these goal yet.

    Crawshaw and I have been going back and forth on how to support struct,
    slice of structs, etc.
    One option we considered is to promote the struct type to its pointer
    type. It seems doable but some may be unhappy about performance.
    Another proposal is to serialize the entire struct object and instantiate
    an object in Java or Objective-C based on the serialized data. It
    eliminates cross-language reference counting.
    But we are not sure about how to handle struct with non-trivial methods
    which may involve unexported fields to work.

    Running out of ideas, I am open to others' ideas...
    I believe, once this is settled, we can start looking into suitable
    encoding/decoding technologies wherever necessary.

    - Hana

    On Tue, Jan 26, 2016 at 6:49 AM, Daniele Baroncelli <
    daniele.b...@gmail.com> wrote:
    The type system in GoMobile is still very limited, as it's currently not
    possibly to work with slices of struct:
    https://godoc.org/golang.org/x/mobile/cmd/gobind#hdr-Type_restrictions

    In the meantime, developers are starting to release mobile apps with
    GoMobile and they all have to deal with the same problem: the lack of
    slices of struct. Each developer has been coming with its own solution.
    Someone tried with wrappers (
    https://github.com/scisci/go-mobile-collection), but the most
    convincing solution so far seems to be Flatbuffers. A few developers have
    already used it successfully.

    Flatbuffers have official implementations in Go and Java. And there is
    already an open source package for Swift:
    https://github.com/mzaks/FlatBuffersSwift

    I have talked to Robert Winslow, one of the guys behind Flatbuffers, if
    it made sense to implement Flatbuffers for GoMobile and he seems to agree
    and he is willing to help.

    I am opening this thread, so that discussions can start. --
    You received this message because you are subscribed to the Google
    Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-dev+...@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.


    --
    __
    --
    You received this message because you are subscribed to the Google Groups
    "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Daniele Baroncelli at Jan 27, 2016 at 6:11 pm
    Hi Daniel,

    This asynchronous usage sounds interesting, but I think most developers
    would just be happy with the Go library functions working synchronously and
    returning fully structured data. Not being able to return structured data,
    it's the biggest limitation for GoMobile at the moment.

    If allowing asynchronous usage creates such complex dependencies, which
    prevents using slices of struct, maybe we should consider a stripped down
    version of GoMobile that at least can provide a fully operational usage
    with synchronous calls.

    Go-compiled libraries providing fully structured data to Android and iOS
    application is already a great achievement. This alone which would attract
    many new developers to GoMobile.

    In summary, there could be two alternative binding tools:
    1) only allowing synchronous calls, but supporting all types
    2) allowing both synchronous and asynchronous calls, but with a limited set
    of types, until a better solution is found


    Daniele




    Il giorno mercoledì 27 gennaio 2016 18:05:59 UTC+1, Daniel Skinner ha
    scritto:
    what would developers use it for
    As Hana mentioned, gobind allows java to implement a go interface. This is
    something I found useful early on by declaring a Broadcaster interface in
    go and in java I implemented the Broadcaster.Stub to facilitate use of
    http://developer.android.com/reference/android/support/v4/content/LocalBroadcastManager.html as
    by company convention.

    It's not clear simply using flatbuffers accomplishes that type of goal.
    Right now, I'm still interested in the use of flatbuffers but this is
    because, as Hana mentioned, there's still a lot to work out with gobind.

    On Wed, Jan 27, 2016 at 10:29 AM Daniele Baroncelli <
    daniele.b...@gmail.com <javascript:>> wrote:
    Hi Hana,

    thanks for taking part in this discussion

    in my opinion, we need to take a step back and answer some basic
    questions:

    - what is the benefit of a GoMobile library?
    - what would developers use it for?
    - which kind of code would it make sense to write in Go and which one in
    Java/Swift?

    In this diagram I try to describe the typical use for GoMobile:
    goo.gl/UkVA9b

    In general I would say the Go part should be treated as a black box (only
    dealing with data and logic), and the threading and callbacks should be
    managed exclusively from the mobile language.

    For this reason, I think the Go library functions should just be called
    syncronously, providing input parameters and expecting just 1 output value
    (to be compatible with Java functions), without complex cross-language
    references. I think the binding should be kept as simple as possible.

    I would treat the Java/Swift -> Go interface as an API interface. The
    data is served by the Go library, and the Java/Swift just queries it,
    without too many interrelations.


    Daniele




    Il giorno mercoledì 27 gennaio 2016 00:07:41 UTC+1, Hyang-Ah Hana Kim ha
    scritto:
    Thanks for bringing up the issue.

    If the goal was to exchange data between two languages, I think use of
    flatbuffers or other equivalent technologies makes sense.
    Byte slices are currently supported, so it's possible to build your idea
    on top of gobind's byte slice support.

    But Gobind is more than just data passing; it offers cross-language
    object references, callbacks, etc.

    For example, Gobind allows Java-side to implement a Go interface, and
    pass the Java object to Go while holding the actual logic in Java.
    Passing a Go struct in Gobind also means that struct's method are also
    available to the other language.
    Maybe some methods assume the Go object maintain a certain state during
    its life time.
    I don't see how the use of Flatbuffers (or the equivalents) can help
    achieving these goal yet.

    Crawshaw and I have been going back and forth on how to support struct,
    slice of structs, etc.
    One option we considered is to promote the struct type to its pointer
    type. It seems doable but some may be unhappy about performance.
    Another proposal is to serialize the entire struct object and
    instantiate an object in Java or Objective-C based on the serialized data.
    It eliminates cross-language reference counting.
    But we are not sure about how to handle struct with non-trivial methods
    which may involve unexported fields to work.

    Running out of ideas, I am open to others' ideas...
    I believe, once this is settled, we can start looking into suitable
    encoding/decoding technologies wherever necessary.

    - Hana

    On Tue, Jan 26, 2016 at 6:49 AM, Daniele Baroncelli <
    daniele.b...@gmail.com> wrote:
    The type system in GoMobile is still very limited, as it's currently not
    possibly to work with slices of struct:
    https://godoc.org/golang.org/x/mobile/cmd/gobind#hdr-Type_restrictions

    In the meantime, developers are starting to release mobile apps with
    GoMobile and they all have to deal with the same problem: the lack of
    slices of struct. Each developer has been coming with its own solution.
    Someone tried with wrappers (
    https://github.com/scisci/go-mobile-collection), but the most
    convincing solution so far seems to be Flatbuffers. A few developers have
    already used it successfully.

    Flatbuffers have official implementations in Go and Java. And there is
    already an open source package for Swift:
    https://github.com/mzaks/FlatBuffersSwift

    I have talked to Robert Winslow, one of the guys behind Flatbuffers, if
    it made sense to implement Flatbuffers for GoMobile and he seems to agree
    and he is willing to help.

    I am opening this thread, so that discussions can start. --
    You received this message because you are subscribed to the Google
    Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send
    an email to golang-dev+...@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.


    --
    __
    --
    You received this message because you are subscribed to the Google Groups
    "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-dev+...@googlegroups.com <javascript:>.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Daniele Baroncelli at Jan 27, 2016 at 6:31 pm

    In summary, there could be two alternative binding tools:
    1) only allowing synchronous calls, but supporting all types
    2) allowing both synchronous and asynchronous calls, but with a limited
    set of types, until a better solution is found

    otherwise, just keeping one binding tool, but supporting slices of struct
    only for synchronous calls?

    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Daniel Skinner at Jan 27, 2016 at 6:32 pm
    I think the point is not that gobind can't move forward with more types due
    to complexity, but rather a determination for how to move forward hasn't
    been made.

    As it stands, any synchronization format that's supported on both the go
    and the java side can be used today with a very light shim. If you're
    willing to compromise your go structs, you can even avoid the shim and
    provide structured data today.

    As for what makes developers happy, one might even argue that an interface
    that provides a single struct value at a given position with fields
    currently supported by gobind already makes them happy when implementing
    long lists, as will be the case when implementing a recycler view adapter
    that responds well to changes of the underlying data within go.

    There's a number of use-cases and serializing large lists of structs is but
    one of them.

    Out of all the android projects I've done, I can't even remember the last
    time I manually managed a list of items in memory to be displayed
    (anecdotal, i know) but for maybe the simplest of cases.

    I'd sooner like to see a determination be made for moving forward with
    gobind that is fast/efficient than a shim I could write on my own in a few
    hours. Sorry if the opinion comes off hard, it's just my stance on the
    matter.

    As an example, and I don't know what the numbers are here, but if go code
    is grabbing an xml feed and deserializes to go structs and then
    reserializes to [insert-serialization-format] to have java deserialize that
    format, why bother with all the step-arounds? Grab the xml and send it
    verbatim to Java and let it deserialize once. The details of this are
    sketchy at best because again it comes down to the use-case and what needs
    to be done, but simply having a serialization format only solves a small
    part of the overall problem.
    On Wed, Jan 27, 2016 at 12:11 PM Daniele Baroncelli wrote:

    Hi Daniel,

    This asynchronous usage sounds interesting, but I think most developers
    would just be happy with the Go library functions working synchronously and
    returning fully structured data. Not being able to return structured data,
    it's the biggest limitation for GoMobile at the moment.

    If allowing asynchronous usage creates such complex dependencies, which
    prevents using slices of struct, maybe we should consider a stripped down
    version of GoMobile that at least can provide a fully operational usage
    with synchronous calls.

    Go-compiled libraries providing fully structured data to Android and iOS
    application is already a great achievement. This alone which would attract
    many new developers to GoMobile.

    In summary, there could be two alternative binding tools:
    1) only allowing synchronous calls, but supporting all types
    2) allowing both synchronous and asynchronous calls, but with a limited
    set of types, until a better solution is found



    Daniele





    Il giorno mercoledì 27 gennaio 2016 18:05:59 UTC+1, Daniel Skinner ha
    scritto:
    what would developers use it for
    As Hana mentioned, gobind allows java to implement a go interface. This
    is something I found useful early on by declaring a Broadcaster interface
    in go and in java I implemented the Broadcaster.Stub to facilitate use of
    http://developer.android.com/reference/android/support/v4/content/LocalBroadcastManager.html as
    by company convention.

    It's not clear simply using flatbuffers accomplishes that type of goal.
    Right now, I'm still interested in the use of flatbuffers but this is
    because, as Hana mentioned, there's still a lot to work out with gobind.
    On Wed, Jan 27, 2016 at 10:29 AM Daniele Baroncelli <
    daniele.b...@gmail.com> wrote:
    Hi Hana,

    thanks for taking part in this discussion

    in my opinion, we need to take a step back and answer some basic
    questions:

    - what is the benefit of a GoMobile library?
    - what would developers use it for?
    - which kind of code would it make sense to write in Go and which one in
    Java/Swift?

    In this diagram I try to describe the typical use for GoMobile:
    goo.gl/UkVA9b

    In general I would say the Go part should be treated as a black box
    (only dealing with data and logic), and the threading and callbacks should
    be managed exclusively from the mobile language.

    For this reason, I think the Go library functions should just be called
    syncronously, providing input parameters and expecting just 1 output value
    (to be compatible with Java functions), without complex cross-language
    references. I think the binding should be kept as simple as possible.

    I would treat the Java/Swift -> Go interface as an API interface. The
    data is served by the Go library, and the Java/Swift just queries it,
    without too many interrelations.


    Daniele




    Il giorno mercoledì 27 gennaio 2016 00:07:41 UTC+1, Hyang-Ah Hana Kim ha
    scritto:
    Thanks for bringing up the issue.

    If the goal was to exchange data between two languages, I think use of
    flatbuffers or other equivalent technologies makes sense.
    Byte slices are currently supported, so it's possible to build your
    idea on top of gobind's byte slice support.

    But Gobind is more than just data passing; it offers cross-language
    object references, callbacks, etc.

    For example, Gobind allows Java-side to implement a Go interface, and
    pass the Java object to Go while holding the actual logic in Java.
    Passing a Go struct in Gobind also means that struct's method are also
    available to the other language.
    Maybe some methods assume the Go object maintain a certain state during
    its life time.
    I don't see how the use of Flatbuffers (or the equivalents) can help
    achieving these goal yet.

    Crawshaw and I have been going back and forth on how to support struct,
    slice of structs, etc.
    One option we considered is to promote the struct type to its pointer
    type. It seems doable but some may be unhappy about performance.
    Another proposal is to serialize the entire struct object and
    instantiate an object in Java or Objective-C based on the serialized data.
    It eliminates cross-language reference counting.
    But we are not sure about how to handle struct with non-trivial methods
    which may involve unexported fields to work.

    Running out of ideas, I am open to others' ideas...
    I believe, once this is settled, we can start looking into suitable
    encoding/decoding technologies wherever necessary.

    - Hana

    On Tue, Jan 26, 2016 at 6:49 AM, Daniele Baroncelli <
    daniele.b...@gmail.com> wrote:
    The type system in GoMobile is still very limited, as it's currently not
    possibly to work with slices of struct:
    https://godoc.org/golang.org/x/mobile/cmd/gobind#hdr-Type_restrictions

    In the meantime, developers are starting to release mobile apps with
    GoMobile and they all have to deal with the same problem: the lack of
    slices of struct. Each developer has been coming with its own solution.
    Someone tried with wrappers (
    https://github.com/scisci/go-mobile-collection), but the most
    convincing solution so far seems to be Flatbuffers. A few developers have
    already used it successfully.

    Flatbuffers have official implementations in Go and Java. And there is
    already an open source package for Swift:
    https://github.com/mzaks/FlatBuffersSwift

    I have talked to Robert Winslow, one of the guys behind Flatbuffers,
    if it made sense to implement Flatbuffers for GoMobile and he seems to
    agree and he is willing to help.

    I am opening this thread, so that discussions can start. --
    You received this message because you are subscribed to the Google
    Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send
    an email to golang-dev+...@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.


    --
    __
    --
    You received this message because you are subscribed to the Google
    Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send
    an email to golang-dev+...@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups
    "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Daniele Baroncelli at Jan 27, 2016 at 7:14 pm

    As an example, and I don't know what the numbers are here, but if go code
    is grabbing an xml feed and deserializes to go structs and then
    reserializes to [insert-serialization-format] to have java deserialize that
    format, why bother with all the step-arounds? Grab the xml and send it
    verbatim to Java and let it deserialize once. The details of this are
    sketchy at best because again it comes down to the use-case and what needs
    to be done, but simply having a serialization format only solves a small
    part of the overall problem.

    The advantage of a serialization protocol like FlatBuffer is that it makes
    no allocation. So it's much faster and more memory efficient.

    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Daniel Skinner at Jan 27, 2016 at 7:21 pm
    this is comparing apples to oranges though, no? This would be comparing an
    xml deserializer in java to a flatbuffer. The underlying mechanism of
    transferring the []byte is still going to cause an allocation (unless
    that's changed recently).
    On Wed, Jan 27, 2016 at 1:14 PM Daniele Baroncelli wrote:


    As an example, and I don't know what the numbers are here, but if go code
    is grabbing an xml feed and deserializes to go structs and then
    reserializes to [insert-serialization-format] to have java deserialize that
    format, why bother with all the step-arounds? Grab the xml and send it
    verbatim to Java and let it deserialize once. The details of this are
    sketchy at best because again it comes down to the use-case and what needs
    to be done, but simply having a serialization format only solves a small
    part of the overall problem.

    The advantage of a serialization protocol like FlatBuffer is that it makes
    no allocation. So it's much faster and more memory efficient.

    --
    You received this message because you are subscribed to the Google Groups
    "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Jason E. Aten at Jan 27, 2016 at 7:28 pm
    If I understand right, no, []byte won't necessarily cause allocation. Note
    that []byte is effectively a pointer, albeit a fat pointer that knows its
    bound and is 24 bytes (3 words) on amd64.

    So the pointer that is []byte can point to off-heap memory/memory mapped
    data, and []byte itself can reside itself on the stack and incur no
    allocation.
    On Wed, Jan 27, 2016 at 11:21 AM, Daniel Skinner wrote:

    this is comparing apples to oranges though, no? This would be comparing an
    xml deserializer in java to a flatbuffer. The underlying mechanism of
    transferring the []byte is still going to cause an allocation (unless
    that's changed recently).
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Daniel Skinner at Jan 27, 2016 at 7:31 pm
    https://github.com/golang/go/issues/12113 i believe currently it copies the
    byte slice, also noted in the gobind docs that []byte doesn't support
    mutation b/c of this.

    On Wed, Jan 27, 2016 at 1:28 PM Jason E. Aten wrote:

    If I understand right, no, []byte won't necessarily cause allocation. Note
    that []byte is effectively a pointer, albeit a fat pointer that knows its
    bound and is 24 bytes (3 words) on amd64.

    So the pointer that is []byte can point to off-heap memory/memory mapped
    data, and []byte itself can reside itself on the stack and incur no
    allocation.

    On Wed, Jan 27, 2016 at 11:21 AM, Daniel Skinner wrote:

    this is comparing apples to oranges though, no? This would be comparing
    an xml deserializer in java to a flatbuffer. The underlying mechanism of
    transferring the []byte is still going to cause an allocation (unless
    that's changed recently).
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Daniel Skinner at Jan 27, 2016 at 7:32 pm
    email sent prematurely... and the docs

    Byte slice types. Note the current implementation does not
       support data mutation of slices passed in as function arguments.

    but is this only for java -> go and not the other way?
    On Wed, Jan 27, 2016 at 1:33 PM Daniel Skinner wrote:

    https://github.com/golang/go/issues/12113 i believe currently it copies
    the byte slice, also noted in the gobind docs that []byte doesn't support
    mutation b/c of this.

    On Wed, Jan 27, 2016 at 1:28 PM Jason E. Aten wrote:

    If I understand right, no, []byte won't necessarily cause allocation.
    Note that []byte is effectively a pointer, albeit a fat pointer that knows
    its bound and is 24 bytes (3 words) on amd64.

    So the pointer that is []byte can point to off-heap memory/memory mapped
    data, and []byte itself can reside itself on the stack and incur no
    allocation.

    On Wed, Jan 27, 2016 at 11:21 AM, Daniel Skinner wrote:

    this is comparing apples to oranges though, no? This would be comparing
    an xml deserializer in java to a flatbuffer. The underlying mechanism of
    transferring the []byte is still going to cause an allocation (unless
    that's changed recently).
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Daniele Baroncelli at Jan 27, 2016 at 7:34 pm
    Jason said it right.

    Here are some interesting Go serialization benchmarks:
    https://github.com/alecthomas/go_serialization_benchmarks/blob/2f5b86e/README.md

    As you can see FlatBuffers is the only serialization with zero allocation.


    Daniele


    Il giorno mercoledì 27 gennaio 2016 20:28:36 UTC+1, Jason E. Aten ha
    scritto:
    If I understand right, no, []byte won't necessarily cause allocation. Note
    that []byte is effectively a pointer, albeit a fat pointer that knows its
    bound and is 24 bytes (3 words) on amd64.

    So the pointer that is []byte can point to off-heap memory/memory mapped
    data, and []byte itself can reside itself on the stack and incur no
    allocation.

    On Wed, Jan 27, 2016 at 11:21 AM, Daniel Skinner <dan...@dasa.cc
    <javascript:>> wrote:
    this is comparing apples to oranges though, no? This would be comparing
    an xml deserializer in java to a flatbuffer. The underlying mechanism of
    transferring the []byte is still going to cause an allocation (unless
    that's changed recently).
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Jason E. Aten at Jan 27, 2016 at 7:39 pm
    Well, those benchmarks are misleading. I pointed out the author how to
    avoid allocations using capnproto, but he declined to follow said advice.

    Daniel pointed out that, although Go in general might not copy for a
    []byte, gomobile is doing copying nonetheless; per
    https://github.com/golang/go/issues/12113
    On Wed, Jan 27, 2016 at 11:34 AM, Daniele Baroncelli wrote:

    Jason said it right.

    Here are some interesting Go serialization benchmarks:

    https://github.com/alecthomas/go_serialization_benchmarks/blob/2f5b86e/README.md

    As you can see FlatBuffers is the only serialization with zero allocation.


    Daniele


    Il giorno mercoledì 27 gennaio 2016 20:28:36 UTC+1, Jason E. Aten ha
    scritto:
    If I understand right, no, []byte won't necessarily cause allocation.
    Note that []byte is effectively a pointer, albeit a fat pointer that knows
    its bound and is 24 bytes (3 words) on amd64.

    So the pointer that is []byte can point to off-heap memory/memory mapped
    data, and []byte itself can reside itself on the stack and incur no
    allocation.
    On Wed, Jan 27, 2016 at 11:21 AM, Daniel Skinner wrote:

    this is comparing apples to oranges though, no? This would be comparing
    an xml deserializer in java to a flatbuffer. The underlying mechanism of
    transferring the []byte is still going to cause an allocation (unless
    that's changed recently).
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Daniel Skinner at Jan 27, 2016 at 7:43 pm
    Yeah, it looks like its copying only for lang -> go
    reads: https://github.com/golang/mobile/blob/master/bind/seq/buffer.go#L131
    writes: https://github.com/golang/mobile/blob/master/bind/seq/buffer.go#L227
    On Wed, Jan 27, 2016 at 1:39 PM Jason E. Aten wrote:

    Well, those benchmarks are misleading. I pointed out the author how to
    avoid allocations using capnproto, but he declined to follow said advice.

    Daniel pointed out that, although Go in general might not copy for a
    []byte, gomobile is doing copying nonetheless; per
    https://github.com/golang/go/issues/12113

    On Wed, Jan 27, 2016 at 11:34 AM, Daniele Baroncelli <
    daniele.baroncelli@gmail.com> wrote:
    Jason said it right.

    Here are some interesting Go serialization benchmarks:

    https://github.com/alecthomas/go_serialization_benchmarks/blob/2f5b86e/README.md

    As you can see FlatBuffers is the only serialization with zero allocation.


    Daniele


    Il giorno mercoledì 27 gennaio 2016 20:28:36 UTC+1, Jason E. Aten ha
    scritto:
    If I understand right, no, []byte won't necessarily cause allocation.
    Note that []byte is effectively a pointer, albeit a fat pointer that knows
    its bound and is 24 bytes (3 words) on amd64.

    So the pointer that is []byte can point to off-heap memory/memory mapped
    data, and []byte itself can reside itself on the stack and incur no
    allocation.
    On Wed, Jan 27, 2016 at 11:21 AM, Daniel Skinner wrote:

    this is comparing apples to oranges though, no? This would be comparing
    an xml deserializer in java to a flatbuffer. The underlying mechanism of
    transferring the []byte is still going to cause an allocation (unless
    that's changed recently).
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Codeprojpub at Jan 29, 2016 at 5:59 am
    Would it require for serialization mechanism on another side to know go
    memory layout? If no, than I don't really understand how that suppose to
    work.

    среда, 27 января 2016 г., 22:28:36 UTC+3 пользователь Jason E. Aten написал:
    If I understand right, no, []byte won't necessarily cause allocation. Note
    that []byte is effectively a pointer, albeit a fat pointer that knows its
    bound and is 24 bytes (3 words) on amd64.

    So the pointer that is []byte can point to off-heap memory/memory mapped
    data, and []byte itself can reside itself on the stack and incur no
    allocation.

    On Wed, Jan 27, 2016 at 11:21 AM, Daniel Skinner <dan...@dasa.cc
    <javascript:>> wrote:
    this is comparing apples to oranges though, no? This would be comparing
    an xml deserializer in java to a flatbuffer. The underlying mechanism of
    transferring the []byte is still going to cause an allocation (unless
    that's changed recently).
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Codeprojpub at Jan 29, 2016 at 5:59 am
    Hey, everyone.

    Current gobind implementation is not ideal, but I think stripping down to
    the synchronous call's only or splitting into two separate types of calling
    would be a bad idea. I can only speak for Android development, tho.

    This implementation allows us to implement entire app runtime inside go,
    which is tremendously useful because android java threading model doesn't
    scale well. Async operations support is lacking, and because there is no
    lambda support - mobile developer should be extremely careful or resources
    start leaking. Several libraries (including the port's from Rx's) attempted
    to fix it, but so far every attempt has some bad sides.

    Right now it's quite easy to pass Java "callbacks" inside go lib, which
    allows Java to have the minimal amount of "non-UI" libraries, which in
    turn, allows focusing on drawing things. instead of figuring out how to get
    them either from source or from go side of an app. Stripping down to
    synchronous only would require implementing some kind of poller in the
    separate thread and a whole lot of new glue code, which is currently
    handled by gobind.

    Another good thing is that current implementation allows us to implement
    state inside go. Not many of mobile developers use it, but go allowed me to
    implement an entire stateful application, with several long-running (I am
    talking about from minutes to hours) background tasks. Some would say, that
    android already provides utilities for that, but the ease of communication
    between tasks and subsequent cancellation is just too good for me to give
    up.

    As for handling calls in two different manners - it would confuse new
    developers and support glue code inside of gobind would be undoubtedly
    harder. Instead of having one problem we will have two.

    My thoughts on that matter are that if we need additional speed
    improvements, perhaps something can be done for cross-language calling
    speed with primitive arguments when the result is a reference to another
    struct? Most of the time, the end UI requires primitive types (ints, floats
    and so on) and strings, so it takes some method chaining (like
    val.GetThings().GetLast().GetVal() for example)to get there. Handling
    complex structures like lists\arrays require some additional go code but it
    comes with almost no additional memory costs. Slices can be quite big, and
    copying their insides for each result will require additional memory, which
    is bad for some of us.

    Dmitry.


    среда, 27 января 2016 г., 22:21:46 UTC+3 пользователь Daniel Skinner
    написал:
    this is comparing apples to oranges though, no? This would be comparing an
    xml deserializer in java to a flatbuffer. The underlying mechanism of
    transferring the []byte is still going to cause an allocation (unless
    that's changed recently).

    On Wed, Jan 27, 2016 at 1:14 PM Daniele Baroncelli <daniele.b...@gmail.com
    <javascript:>> wrote:
    As an example, and I don't know what the numbers are here, but if go code
    is grabbing an xml feed and deserializes to go structs and then
    reserializes to [insert-serialization-format] to have java deserialize that
    format, why bother with all the step-arounds? Grab the xml and send it
    verbatim to Java and let it deserialize once. The details of this are
    sketchy at best because again it comes down to the use-case and what needs
    to be done, but simply having a serialization format only solves a small
    part of the overall problem.

    The advantage of a serialization protocol like FlatBuffer is that it
    makes no allocation. So it's much faster and more memory efficient.

    --
    You received this message because you are subscribed to the Google Groups
    "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-dev+...@googlegroups.com <javascript:>.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Sridhar at Jan 27, 2016 at 4:43 pm
    As Hana pointed out, I don't know if mandating flatbuffers, etc works with
    how reference counting is implemented in gomobile currently.

    With my current work, the following approach is proving to be useful:

    1. Copy everything unless specified. Every object does not need to be
    reference counted. Both iOS and Android have a defined application
    lifecycle. Given this, it is possible to turn most calls across the
    boundary into copies with reference counting being restricted to just an
    explicit subset of objects.

    2. Following from the above, treat the runtime boundary as an
    bi-directional RPC layer. There are RPC services where the service objects
    are reference counted, but all method parameters and return values are
    copied. Service object references cannot be passed in methods. Services can
    be exposed by Java/ObjC/Swift/Go.

    My understanding is that the above approach doesn't really fit with
    gomobile bind's current approach since it aims to be a proper binding layer
    and not merely an rpc layer. However, in practice I have found the rpc
    approach easier to reason about.

    Sridhar


    On Wednesday, January 27, 2016 at 12:07:41 AM UTC+1, Hyang-Ah Hana Kim
    wrote:
    Thanks for bringing up the issue.

    If the goal was to exchange data between two languages, I think use of
    flatbuffers or other equivalent technologies makes sense.
    Byte slices are currently supported, so it's possible to build your idea
    on top of gobind's byte slice support.

    But Gobind is more than just data passing; it offers cross-language object
    references, callbacks, etc.

    For example, Gobind allows Java-side to implement a Go interface, and pass
    the Java object to Go while holding the actual logic in Java.
    Passing a Go struct in Gobind also means that struct's method are also
    available to the other language.
    Maybe some methods assume the Go object maintain a certain state during
    its life time.
    I don't see how the use of Flatbuffers (or the equivalents) can help
    achieving these goal yet.

    Crawshaw and I have been going back and forth on how to support struct,
    slice of structs, etc.
    One option we considered is to promote the struct type to its pointer
    type. It seems doable but some may be unhappy about performance.
    Another proposal is to serialize the entire struct object and instantiate
    an object in Java or Objective-C based on the serialized data. It
    eliminates cross-language reference counting.
    But we are not sure about how to handle struct with non-trivial methods
    which may involve unexported fields to work.

    Running out of ideas, I am open to others' ideas...
    I believe, once this is settled, we can start looking into suitable
    encoding/decoding technologies wherever necessary.

    - Hana


    On Tue, Jan 26, 2016 at 6:49 AM, Daniele Baroncelli <
    daniele.b...@gmail.com <javascript:>> wrote:
    The type system in GoMobile is still very limited, as it's currently not
    possibly to work with slices of struct:
    https://godoc.org/golang.org/x/mobile/cmd/gobind#hdr-Type_restrictions

    In the meantime, developers are starting to release mobile apps with
    GoMobile and they all have to deal with the same problem: the lack of
    slices of struct. Each developer has been coming with its own solution.
    Someone tried with wrappers (
    https://github.com/scisci/go-mobile-collection), but the most convincing
    solution so far seems to be Flatbuffers. A few developers have already used
    it successfully.

    Flatbuffers have official implementations in Go and Java. And there is
    already an open source package for Swift:
    https://github.com/mzaks/FlatBuffersSwift

    I have talked to Robert Winslow, one of the guys behind Flatbuffers, if
    it made sense to implement Flatbuffers for GoMobile and he seems to agree
    and he is willing to help.

    I am opening this thread, so that discussions can start.

    --
    You received this message because you are subscribed to the Google Groups
    "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-dev+...@googlegroups.com <javascript:>.
    For more options, visit https://groups.google.com/d/optout.


    --
    __
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • J E Aten at Jan 27, 2016 at 5:07 pm
    I've had quite a bit of experience in this area.

    Briefly: for language interop, the easiest thing to do is to use the
    excellent support
    for JSON and msgpack that Go libraries already provides.

    For writing to disk, write one of these in parallel alongside your binary
    data. For transmitting over the wire, use
    the builtin JSON or 3rd party libraries (e.g. ugorji/go/codec or
    tinylib/msgp) to translate from your Go structs on the fly.

    More in depth thoughts:

    From investigation of flatbuffers last month:

    The current state is that the Go bindings for FlatBuffers are buggy and
    incomplete;
    it wasn't possible to produce equivalent binary output to the C++ reference
    implementation
    with the Go bindings.

    The design of FlatBuffers is also, to my mind, poorly thought out: it is
    such that
    there are severe limitations on the amount of data you can express;
    you are limited to 64KB + 1 longer array. The
    documentation is very incomplete; there is no precise spec, rather just a
    few
    paragraphs of vague casual description that do not match hex dumps
    of the reference implementation output. As a result the format is currently
    so
    under-documented that I cannot recommend it.

    Summary of flatbuffers investigation: I would like to see the Go bindings
    improve,
    but the originators need to put in better documentation of the format
    before that is even possible.

    disclaimer: I maintain the first version of the Go bindings for Capnproto.

    For capnroto, on the plus side:

    + The serialization design/IDL for Capnproto is generally excellent. Very
    expressive and well designed type system and solid evolve-able story.

    - But since you talk about having arrays of structs, be aware the in
    capnproto,
    you will have no ability to version such structs; such structs are always
    stored in-line in the array. Hence the advantages
    of using an evolve-able data format are lost. This begs the question
    of why go to the trouble in the first place.

    - Support for languages beyond C++, python, and Go is unlikely to be
    completed in the next year.

    More meta:

    For most new projects I've moved to the simple Msgpack
    and using either https://github.com/tinylib/msgp or
    https://github.com/ugorji/go,
    with the need for easy json interop being forefront in my mind.

    Between those two libraries you have alot of speed tuning options (either
    fully dynamic (ugorgi), or fully pre-compiled deserialization (tinylib).

    The caveats with Msgpack are poor handling of signed vs unsigned
    numbers; the lack standard ways to express time.Time and other
    language specific types, and the fact that although they tried to
    fix the flaws of version1 with version2, nobody really implemented
    version2 fully. Also the evolution of data story is not as bullet-proof;
    actual interop with other languages may be particularly
    painful because of that. That said, for an all-Go project,
    using tinylib/msgp in particular makes serialization so fast and
    easy (add a "go generate" one-line directive to your file) that it is my
    default solution. You get quite a bit of ability to evolve your
    data, as long as you never re-use an old field name and give
    it a different type. It is somewhat more risky, but as long
    as you only care about version N to version N+1 evolve-abilty,
    you should be fine.

    Stepping back, big picture: Protobufs with GoGoProtobuf
    certainly has the most mature tooling and support of any of these.
    If you don't want to pay bleeding edge "early adopter" taxes,
    and you require a binary format, this is your choice.

    On Tuesday, January 26, 2016 at 5:29:41 AM UTC-8, Daniele Baroncelli wrote:

    The type system in GoMobile is still very limited, as it's currently not
    possibly to work with slices of struct:
    https://godoc.org/golang.org/x/mobile/cmd/gobind#hdr-Type_restrictions

    In the meantime, developers are starting to release mobile apps with
    GoMobile and they all have to deal with the same problem: the lack of
    slices of struct. Each developer has been coming with its own solution.
    Someone tried with wrappers (
    https://github.com/scisci/go-mobile-collection), but the most convincing
    solution so far seems to be Flatbuffers. A few developers have already used
    it successfully.

    Flatbuffers have official implementations in Go and Java. And there is
    already an open source package for Swift:
    https://github.com/mzaks/FlatBuffersSwift

    I have talked to Robert Winslow, one of the guys behind Flatbuffers, if it
    made sense to implement Flatbuffers for GoMobile and he seems to agree and
    he is willing to help.

    I am opening this thread, so that discussions can start.
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Seb Binet at Jan 27, 2016 at 5:56 pm

    On Wed, Jan 27, 2016 at 4:32 PM, wrote:
    I've had quite a bit of experience in this area.

    Briefly: for language interop, the easiest thing to do is to use the
    excellent support
    for JSON and msgpack that Go libraries already provides.

    For writing to disk, write one of these in parallel alongside your binary
    data. For transmitting over the wire, use
    the builtin JSON or 3rd party libraries (e.g. ugorji/go/codec or
    tinylib/msgp) to translate from your Go structs on the fly.

    More in depth thoughts:

    From investigation of flatbuffers last month:

    The current state is that the Go bindings for FlatBuffers are buggy and
    incomplete;
    it wasn't possible to produce equivalent binary output to the C++ reference
    implementation
    with the Go bindings.

    The design of FlatBuffers is also, to my mind, poorly thought out: it is
    such that
    there are severe limitations on the amount of data you can express;
    you are limited to 64KB + 1 longer array. The
    documentation is very incomplete; there is no precise spec, rather just a
    few
    paragraphs of vague casual description that do not match hex dumps
    of the reference implementation output. As a result the format is currently
    so
    under-documented that I cannot recommend it.

    Summary of flatbuffers investigation: I would like to see the Go bindings
    improve,
    but the originators need to put in better documentation of the format
    before that is even possible.

    disclaimer: I maintain the first version of the Go bindings for Capnproto.

    For capnroto, on the plus side:

    + The serialization design/IDL for Capnproto is generally excellent. Very
    expressive and well designed type system and solid evolve-able story.

    - But since you talk about having arrays of structs, be aware the in
    capnproto,
    you will have no ability to version such structs; such structs are always
    stored in-line in the array. Hence the advantages
    of using an evolve-able data format are lost. This begs the question
    of why go to the trouble in the first place.

    - Support for languages beyond C++, python, and Go is unlikely to be
    completed in the next year.

    More meta:

    For most new projects I've moved to the simple Msgpack
    and using either https://github.com/tinylib/msgp or
    https://github.com/ugorji/go,
    with the need for easy json interop being forefront in my mind.

    Between those two libraries you have alot of speed tuning options (either
    fully dynamic (ugorgi), or fully pre-compiled deserialization (tinylib).

    The caveats with Msgpack are poor handling of signed vs unsigned
    numbers; the lack standard ways to express time.Time and other
    language specific types, and the fact that although they tried to
    fix the flaws of version1 with version2, nobody really implemented
    version2 fully. Also the evolution of data story is not as bullet-proof;
    actual interop with other languages may be particularly
    painful because of that. That said, for an all-Go project,
    using tinylib/msgp in particular makes serialization so fast and
    easy (add a "go generate" one-line directive to your file) that it is my
    default solution. You get quite a bit of ability to evolve your
    data, as long as you never re-use an old field name and give
    it a different type. It is somewhat more risky, but as long
    as you only care about version N to version N+1 evolve-abilty,
    you should be fine.
    a naive question: wouldn't CBOR be a valid alternative?
    it's JSON but binary and standardized, isn't it?
    and there are implementations in various languages.

    -s

    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Nicetan at Jan 28, 2016 at 4:33 am
    Hopefully this isn't too off topic, but for those interested strictly in a
    cross-platform protocol binding, you should really look at what keybase is
    using for their client
    (https://keybase.io/docs/client/client_architecture#rpc-protocol). You
    basically define your protocol in AVDL syntax
    (https://github.com/keybase/client/tree/master/protocol/avdl) and then are
    able to build it for ios, android, javascript, and go. I look at gomobile
    as being more than just a protocol binding obviously, but thought this
    might interest some.

    Dan
    On Tuesday, January 26, 2016 at 8:29:41 AM UTC-5, Daniele Baroncelli wrote:

    The type system in GoMobile is still very limited, as it's currently not
    possibly to work with slices of struct:
    https://godoc.org/golang.org/x/mobile/cmd/gobind#hdr-Type_restrictions

    In the meantime, developers are starting to release mobile apps with
    GoMobile and they all have to deal with the same problem: the lack of
    slices of struct. Each developer has been coming with its own solution.
    Someone tried with wrappers (
    https://github.com/scisci/go-mobile-collection), but the most convincing
    solution so far seems to be Flatbuffers. A few developers have already used
    it successfully.

    Flatbuffers have official implementations in Go and Java. And there is
    already an open source package for Swift:
    https://github.com/mzaks/FlatBuffersSwift

    I have talked to Robert Winslow, one of the guys behind Flatbuffers, if it
    made sense to implement Flatbuffers for GoMobile and he seems to agree and
    he is willing to help.

    I am opening this thread, so that discussions can start.
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-dev @
categoriesgo
postedJan 26, '16 at 1:29p
activeJan 29, '16 at 5:59a
posts25
users10
websitegolang.org

People

Translate

site design / logo © 2021 Grokbase