FAQ
Hello,

Let's say I have the following simple struct:

type MyStruct struct {
data string
}

And there is simple Writer for it:

func (s *MyStruct) Write(p []byte) (n int, err error) {
s.data = string(p)
return len(p), nil
}

It assumes that p contains all the data, it's not possible to write a part of struct. We can call it "block-oriented" Writer. This is how it can be used:

s := MyStruct{}
r := strings.NewReader("123")
p := make([]byte, 42)
for {
_, err := r.Read(p)
if err == io.EOF {
break
}
s.Write(p)
}

However, this Writer doesn't play nice with "stream-oriented" Readers. If we use iotest.OneByteReader or just decrease p size to 1, then s will contain only one last byte.

Reader for struct shares this problem. So the question is: does it make sense to provide Reader and Writer for a struct? Or it's better to provide custom marshaler and unmarshaler and clearly document them as "block-oriented"?

–-–
Alexey Palazhchenko

Search Discussions

  • Peter S at Sep 9, 2012 at 4:27 am
    Marshaling should be done independently from Reader/Writer.

    The definition for Write in io.Writer is: "Write writes len(p) bytes from p
    to the underlying data stream." First, this means that Writers (as well as
    Readers) are "stream-oriented." Second, what you are trying to do
    (marshaling / unmarshaling) does not fit this definition, which means you
    shouldn't be doing it in Write / Read. (As you correctly noticed, doing so
    will cause problems when interacting with standard and third party APIs
    using Writers and Readers.)

    If you are trying to exchange data between Go programs only, you should
    probably look at the encoding/gob
    <http://golang.org/pkg/encoding/gob/>package which supports both
    marshaling and streaming. You can use a
    gob.Encoder / gob.Decoder to send/receive data through any stream
    (Reader/Writer), so your code won't have to take care of fragmentation at
    the stream level. You can also define custom marshaling for your types by
    implementing the GobEncoder and GobDecoder interfaces.

    If you are trying to exchange data between Go and non-Go programs, you
    should probably look at some of the other common encodings such as XML,
    JSON, or protobuf (unless you want to reinvent the wheel).

    Peter

    On Sun, Sep 9, 2012 at 6:26 AM, Alexey Palazhchenko wrote:

    Hello,

    Let's say I have the following simple struct:

    type MyStruct struct {
    data string
    }

    And there is simple Writer for it:

    func (s *MyStruct) Write(p []byte) (n int, err error) {
    s.data = string(p)
    return len(p), nil
    }

    It assumes that p contains all the data, it's not possible to write a part
    of struct. We can call it "block-oriented" Writer. This is how it can be
    used:

    s := MyStruct{}
    r := strings.NewReader("123")
    p := make([]byte, 42)
    for {
    _, err := r.Read(p)
    if err == io.EOF {
    break
    }
    s.Write(p)
    }

    However, this Writer doesn't play nice with "stream-oriented" Readers. If
    we use iotest.OneByteReader or just decrease p size to 1, then s will
    contain only one last byte.

    Reader for struct shares this problem. So the question is: does it make
    sense to provide Reader and Writer for a struct? Or it's better to provide
    custom marshaler and unmarshaler and clearly document them as
    "block-oriented"?

    –-–
    Alexey Palazhchenko
  • Kyle Lemons at Sep 10, 2012 at 8:53 pm
    I think you're looking for ReadFrom/WriteTo, not Write/Read, if you want to
    implement a "standard" interface with your type with similar semantics.
    On Sat, Sep 8, 2012 at 2:26 PM, Alexey Palazhchenko wrote:

    Hello,

    Let's say I have the following simple struct:

    type MyStruct struct {
    data string
    }

    And there is simple Writer for it:

    func (s *MyStruct) Write(p []byte) (n int, err error) {
    s.data = string(p)
    return len(p), nil
    }

    It assumes that p contains all the data, it's not possible to write a part
    of struct. We can call it "block-oriented" Writer. This is how it can be
    used:

    s := MyStruct{}
    r := strings.NewReader("123")
    p := make([]byte, 42)
    for {
    _, err := r.Read(p)
    if err == io.EOF {
    break
    }
    s.Write(p)
    }

    However, this Writer doesn't play nice with "stream-oriented" Readers. If
    we use iotest.OneByteReader or just decrease p size to 1, then s will
    contain only one last byte.

    Reader for struct shares this problem. So the question is: does it make
    sense to provide Reader and Writer for a struct? Or it's better to provide
    custom marshaler and unmarshaler and clearly document them as
    "block-oriented"?

    –-–
    Alexey Palazhchenko
  • Alexey Palazhchenko at Sep 11, 2012 at 12:12 pm
    Hello,
    I think you're looking for ReadFrom/WriteTo, not Write/Read, if you want to implement a "standard" interface with your type with similar semantics.
    Thanks, looks good.

    –-–
    Alexey Palazhchenko

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedSep 9, '12 at 1:35a
activeSep 11, '12 at 12:12p
posts4
users3
websitegolang.org

People

Translate

site design / logo © 2022 Grokbase