FAQ
Hi,

From gob documentation, it looks like gobs are most suited when bound with
network connection and can be efficient in transmitting data.

I am trying to do something like this
- https://www.lucidchart.com/documents/edit/37f40689-210e-4c8b-abd0-285a46df9629
Application F, is responsible for populating cache, and it encodes data
using gob and sets into cache.
Application A, have to fetch data from cache and decode it and use it.

Now questions are -
1. when application A is restarted - it will not be able to de-serialize
data from cache, as it wont have type information (as its send only in
first gob for respective type?. Please confirm.)
2. I cannot have multiple decoders in application A, because I am not aware
of which objects have type information or not. (Though I think one decoder
will be fine in my case, but just wanted to raise concern).

Structures I am using in F and A are same, I was wondering if we have any
api, which can allow to let gob know, what structures it may expect or
something like Registration while booting application?

Only viable option at this point, looks to me to be have a new gob encoder
every time I encode structure in F. But this I think will be fairly
inefficient.

Please advise.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Search Discussions

  • Siddon Tang at Dec 17, 2014 at 3:11 pm
    Hi

    I met the same problem when using gob with rabbitmq (a publisher publishes
    gob encoding message, multi consumers fetch and decode).

    As we know, a gob stream is self-describing, some type description will be
    added first, then the typeid will be used later. If restarted, the later
    messages only have typeid which you can't decode correctly.

    We have to use a new encoder every time too, this is not efficient but
    correct.

    I hope gob can support global typeid map, not only in a stream. If two
    machines use the same code for gob type register, they may have the same
    typeid map, then they can pass typeid only.

    Btw, now I try to do this, e.g, F and A as yours, and struct T.

    1, F uses gob.Register(&T{})
    2. F new a gob encoder and encodes a empty T into a file as the T type
    proto, like google proto file.
    3. When A new a gob decoder, it must decode T proto file first, so that it
    can handle the message only has typeid.


    在 2014年12月13日星期六UTC+8上午2时24分10秒,suraj写道:
    Hi,

    From gob documentation, it looks like gobs are most suited when bound with
    network connection and can be efficient in transmitting data.

    I am trying to do something like this -
    https://www.lucidchart.com/documents/edit/37f40689-210e-4c8b-abd0-285a46df9629
    Application F, is responsible for populating cache, and it encodes data
    using gob and sets into cache.
    Application A, have to fetch data from cache and decode it and use it.

    Now questions are -
    1. when application A is restarted - it will not be able to de-serialize
    data from cache, as it wont have type information (as its send only in
    first gob for respective type?. Please confirm.)
    2. I cannot have multiple decoders in application A, because I am not
    aware of which objects have type information or not. (Though I think one
    decoder will be fine in my case, but just wanted to raise concern).

    Structures I am using in F and A are same, I was wondering if we have any
    api, which can allow to let gob know, what structures it may expect or
    something like Registration while booting application?

    Only viable option at this point, looks to me to be have a new gob encoder
    every time I encode structure in F. But this I think will be fairly
    inefficient.

    Please advise.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Suraj at Dec 17, 2014 at 6:45 pm

    On Wednesday, December 17, 2014 7:11:41 AM UTC-8, Siddon Tang wrote:
    Hi

    I met the same problem when using gob with rabbitmq (a publisher publishes
    gob encoding message, multi consumers fetch and decode).

    As we know, a gob stream is self-describing, some type description will be
    added first, then the typeid will be used later. If restarted, the later
    messages only have typeid which you can't decode correctly.

    We have to use a new encoder every time too, this is not efficient but
    correct.

    I hope gob can support global typeid map, not only in a stream. If two
    machines use the same code for gob type register, they may have the same
    typeid map, then they can pass typeid only.

    Btw, now I try to do this, e.g, F and A as yours, and struct T.

    1, F uses gob.Register(&T{})
    2. F new a gob encoder and encodes a empty T into a file as the T type
    proto, like google proto file.
    3. When A new a gob decoder, it must decode T proto file first, so that it
    can handle the message only has typeid.

    Excellent Suggestion. Will try this.
    在 2014年12月13日星期六UTC+8上午2时24分10秒,suraj写道:
    Hi,

    From gob documentation, it looks like gobs are most suited when bound
    with network connection and can be efficient in transmitting data.

    I am trying to do something like this -
    https://www.lucidchart.com/documents/edit/37f40689-210e-4c8b-abd0-285a46df9629
    Application F, is responsible for populating cache, and it encodes data
    using gob and sets into cache.
    Application A, have to fetch data from cache and decode it and use it.

    Now questions are -
    1. when application A is restarted - it will not be able to de-serialize
    data from cache, as it wont have type information (as its send only in
    first gob for respective type?. Please confirm.)
    2. I cannot have multiple decoders in application A, because I am not
    aware of which objects have type information or not. (Though I think one
    decoder will be fine in my case, but just wanted to raise concern).

    Structures I am using in F and A are same, I was wondering if we have any
    api, which can allow to let gob know, what structures it may expect or
    something like Registration while booting application?

    Only viable option at this point, looks to me to be have a new gob
    encoder every time I encode structure in F. But this I think will be fairly
    inefficient.

    Please advise.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Suraj at Dec 17, 2014 at 6:48 pm
    Excellent suggestion. Will try writing to file approach.
    One concern is - is typeid constant across restarts for F? If its not, then
    may be we will have issue if F is restarted?
    On Wednesday, December 17, 2014 7:11:41 AM UTC-8, Siddon Tang wrote:

    Hi

    I met the same problem when using gob with rabbitmq (a publisher publishes
    gob encoding message, multi consumers fetch and decode).

    As we know, a gob stream is self-describing, some type description will be
    added first, then the typeid will be used later. If restarted, the later
    messages only have typeid which you can't decode correctly.

    We have to use a new encoder every time too, this is not efficient but
    correct.

    I hope gob can support global typeid map, not only in a stream. If two
    machines use the same code for gob type register, they may have the same
    typeid map, then they can pass typeid only.

    Btw, now I try to do this, e.g, F and A as yours, and struct T.

    1, F uses gob.Register(&T{})
    2. F new a gob encoder and encodes a empty T into a file as the T type
    proto, like google proto file.
    3. When A new a gob decoder, it must decode T proto file first, so that it
    can handle the message only has typeid.


    在 2014年12月13日星期六UTC+8上午2时24分10秒,suraj写道:
    Hi,

    From gob documentation, it looks like gobs are most suited when bound
    with network connection and can be efficient in transmitting data.

    I am trying to do something like this -
    https://www.lucidchart.com/documents/edit/37f40689-210e-4c8b-abd0-285a46df9629
    Application F, is responsible for populating cache, and it encodes data
    using gob and sets into cache.
    Application A, have to fetch data from cache and decode it and use it.

    Now questions are -
    1. when application A is restarted - it will not be able to de-serialize
    data from cache, as it wont have type information (as its send only in
    first gob for respective type?. Please confirm.)
    2. I cannot have multiple decoders in application A, because I am not
    aware of which objects have type information or not. (Though I think one
    decoder will be fine in my case, but just wanted to raise concern).

    Structures I am using in F and A are same, I was wondering if we have any
    api, which can allow to let gob know, what structures it may expect or
    something like Registration while booting application?

    Only viable option at this point, looks to me to be have a new gob
    encoder every time I encode structure in F. But this I think will be fairly
    inefficient.

    Please advise.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Siddon Tang at Dec 18, 2014 at 12:57 am
    Hi

    If you keep the register sequence, e.g register type A, B, C, all the time,
    the typeid may be the same. gob uses 64 for the first user type id, if not
    changed, I think it is ok.

    You can do more test to confirm it.

    在 2014年12月18日星期四UTC+8上午2时48分04秒,suraj写道:
    Excellent suggestion. Will try writing to file approach.
    One concern is - is typeid constant across restarts for F? If its not,
    then may be we will have issue if F is restarted?
    On Wednesday, December 17, 2014 7:11:41 AM UTC-8, Siddon Tang wrote:

    Hi

    I met the same problem when using gob with rabbitmq (a publisher
    publishes gob encoding message, multi consumers fetch and decode).

    As we know, a gob stream is self-describing, some type description will
    be added first, then the typeid will be used later. If restarted, the later
    messages only have typeid which you can't decode correctly.

    We have to use a new encoder every time too, this is not efficient but
    correct.

    I hope gob can support global typeid map, not only in a stream. If two
    machines use the same code for gob type register, they may have the same
    typeid map, then they can pass typeid only.

    Btw, now I try to do this, e.g, F and A as yours, and struct T.

    1, F uses gob.Register(&T{})
    2. F new a gob encoder and encodes a empty T into a file as the T type
    proto, like google proto file.
    3. When A new a gob decoder, it must decode T proto file first, so that
    it can handle the message only has typeid.


    在 2014年12月13日星期六UTC+8上午2时24分10秒,suraj写道:
    Hi,

    From gob documentation, it looks like gobs are most suited when bound
    with network connection and can be efficient in transmitting data.

    I am trying to do something like this -
    https://www.lucidchart.com/documents/edit/37f40689-210e-4c8b-abd0-285a46df9629
    Application F, is responsible for populating cache, and it encodes data
    using gob and sets into cache.
    Application A, have to fetch data from cache and decode it and use it.

    Now questions are -
    1. when application A is restarted - it will not be able to de-serialize
    data from cache, as it wont have type information (as its send only in
    first gob for respective type?. Please confirm.)
    2. I cannot have multiple decoders in application A, because I am not
    aware of which objects have type information or not. (Though I think one
    decoder will be fine in my case, but just wanted to raise concern).

    Structures I am using in F and A are same, I was wondering if we have
    any api, which can allow to let gob know, what structures it may expect or
    something like Registration while booting application?

    Only viable option at this point, looks to me to be have a new gob
    encoder every time I encode structure in F. But this I think will be fairly
    inefficient.

    Please advise.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Greg Roseberry at Dec 18, 2014 at 8:24 am
    Why not using something like this if you're worried about
    speed? https://github.com/philhofer/msgp
    Or just use encoding/json otherwise. Trying to sync up gob type IDs is
    going to be a lot more brittle and annoying than it's worth, IMO.
    On Saturday, December 13, 2014 3:24:10 AM UTC+9, suraj wrote:

    Hi,

    From gob documentation, it looks like gobs are most suited when bound with
    network connection and can be efficient in transmitting data.

    I am trying to do something like this -
    https://www.lucidchart.com/documents/edit/37f40689-210e-4c8b-abd0-285a46df9629
    Application F, is responsible for populating cache, and it encodes data
    using gob and sets into cache.
    Application A, have to fetch data from cache and decode it and use it.

    Now questions are -
    1. when application A is restarted - it will not be able to de-serialize
    data from cache, as it wont have type information (as its send only in
    first gob for respective type?. Please confirm.)
    2. I cannot have multiple decoders in application A, because I am not
    aware of which objects have type information or not. (Though I think one
    decoder will be fine in my case, but just wanted to raise concern).

    Structures I am using in F and A are same, I was wondering if we have any
    api, which can allow to let gob know, what structures it may expect or
    something like Registration while booting application?

    Only viable option at this point, looks to me to be have a new gob encoder
    every time I encode structure in F. But this I think will be fairly
    inefficient.

    Please advise.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedDec 12, '14 at 6:24p
activeDec 18, '14 at 8:24a
posts6
users3
websitegolang.org

People

Translate

site design / logo © 2022 Grokbase