FAQ
Dear all,

Let's strait to the question:

Suppose I want to read something on a Reader and make the Read call
return after a certain time period. I know the net.Conn has
SetDeadline(), but I want to apply it to generic Reader, i.e. io.Reader.

There are already some people proposed their approach long time ago and
it should not be hard. Here is my code:

http://play.golang.org/p/itXsEEseRM

Main part:


func (self *TimeoutReader) Read(buf []byte) (n int, err error) {
ch := make(chan bool)
n = 0
err = nil
go func() {
n, err = self.reader.Read(buf)
ch <- true
}()
select {
case <-ch:
return
case <-time.After(self.timeout):
return 0, errors.New("Timeout")
}
return
}

I won't explain too much on it. It's easy and you get the idea.

However, this code will cause the goroutine created in Read() always run
if the Read() did not return.

Is there any other way to make the goroutine stop running? (I think it
turns to be another old problem on this list: How to kill a goroutine
externally. Well.. I know the answer is: No, we cannot do it.)

Anyway, here is a possible solution I can image:

Test if the reader has a method named SetReadDeadline() or SetDeadline()
by defining new interfaces. If it does, use it to set timeout.

Does anyone has other (better) solutions?

Regards,
-Monnand

--

Search Discussions

  • Maxim Khitrov at Oct 2, 2012 at 5:48 pm

    On Tue, Oct 2, 2012 at 12:29 PM, Monnand wrote:
    Dear all,

    Let's strait to the question:

    Suppose I want to read something on a Reader and make the Read call return
    after a certain time period. I know the net.Conn has SetDeadline(), but I
    want to apply it to generic Reader, i.e. io.Reader.

    There are already some people proposed their approach long time ago and it
    should not be hard. Here is my code:

    http://play.golang.org/p/itXsEEseRM

    I won't explain too much on it. It's easy and you get the idea.

    However, this code will cause the goroutine created in Read() always run if
    the Read() did not return.

    Is there any other way to make the goroutine stop running? (I think it turns
    to be another old problem on this list: How to kill a goroutine externally.
    Well.. I know the answer is: No, we cannot do it.)

    Anyway, here is a possible solution I can image:

    Test if the reader has a method named SetReadDeadline() or SetDeadline() by
    defining new interfaces. If it does, use it to set timeout.

    Does anyone has other (better) solutions?
    You can't stop the goroutine, but you can continue using it in
    subsequent Read calls. Here's my (completely untested) solution, which
    is based on the timeout logic I'm using in my IMAP library [1]:

    http://play.golang.org/p/CX5ojtTOm4

    My library makes one optimization in that the goroutine is always
    running, but then you need a second control channel to issue read
    requests.

    - Max

    [1] https://code.google.com/p/go-imap/source/browse/go1/imap/client.go#347

    --
  • Monnand at Oct 2, 2012 at 7:16 pm
    On 10/02/2012 01:47 PM, Maxim Khitrov wrote:
    <snip>
    You can't stop the goroutine, but you can continue using it in
    subsequent Read calls. Here's my (completely untested) solution, which
    is based on the timeout logic I'm using in my IMAP library [1]:

    http://play.golang.org/p/CX5ojtTOm4

    My library makes one optimization in that the goroutine is always
    running, but then you need a second control channel to issue read
    requests.

    - Max

    [1] https://code.google.com/p/go-imap/source/browse/go1/imap/client.go#347
    Thank you, Max.

    A question on your IMAP library: Why not use SetDeadline in the recv()
    method?

    Regards,
    -Nan

    --
  • Maxim Khitrov at Oct 2, 2012 at 6:54 pm

    On Tue, Oct 2, 2012 at 2:13 PM, Monnand wrote:
    On 10/02/2012 01:47 PM, Maxim Khitrov wrote:
    <snip>
    You can't stop the goroutine, but you can continue using it in
    subsequent Read calls. Here's my (completely untested) solution, which
    is based on the timeout logic I'm using in my IMAP library [1]:

    http://play.golang.org/p/CX5ojtTOm4

    My library makes one optimization in that the goroutine is always
    running, but then you need a second control channel to issue read
    requests.

    - Max

    [1] https://code.google.com/p/go-imap/source/browse/go1/imap/client.go#347
    Thank you, Max.

    A question on your IMAP library: Why not use SetDeadline in the recv()
    method?

    Regards,
    -Nan
    The data pipeline for an IMAP connection looks like this:

    bufio <-> [flate] <-> [tls] <-> net.Conn

    The two middle stages (compression and encryption) are optional.
    Through some testing I found that the flate package doesn't play well
    with temporary errors. After a single timeout, the connection became
    unusable. As a result, I needed a timeout mechanism that bypassed the
    entire receive pipeline.

    I use SetDeadline when initiating and closing the connection. The
    goroutine solution seems safer while the connection is active. It lets
    me treat any error from net.Conn as fatal.

    - Max

    --
  • Monnand at Oct 5, 2012 at 3:10 am
    On 10/02/2012 02:30 PM, Maxim Khitrov wrote:
    <snip>
    The data pipeline for an IMAP connection looks like this:

    bufio <-> [flate] <-> [tls] <-> net.Conn

    The two middle stages (compression and encryption) are optional.
    Through some testing I found that the flate package doesn't play well
    with temporary errors. After a single timeout, the connection became
    unusable. As a result, I needed a timeout mechanism that bypassed the
    entire receive pipeline.

    I use SetDeadline when initiating and closing the connection. The
    goroutine solution seems safer while the connection is active. It lets
    me treat any error from net.Conn as fatal.

    - Max
    Let me see if I get it correct: you said "After a single timeout, the
    connection became unusable." So that means, if a Read/Write() returns a
    timeout error because of setting SetDeadline, I can no longer use the
    whole connection, am I correct?

    This seems not reasonable. I mean the behavior of the library should not
    be like this. Is this a feature or a bug of the standard library? Can
    you reproduce this scenario on every timeout?

    Actually, after sending the first mail, I thought using SetDeadline()
    would be a better way in sense of performance and safety. The argument I
    made was:

    Timeout is a feature of the Writer/Reader. Some Writer/Reader may not
    provide such feature. Imagine the Writer/Reader is a hardware, if they
    do not support timeout, simply stop the goroutine externally (suppose we
    can do that) may not be a good idea. In conclusion, the timeout feature
    should be provided by the Reader/Writer.

    However, if it is as you said, one single timeout would invalidate the
    whole connection, then it won't make sense to use SetDeadline().

    Regards,
    -Nan

    --
  • Maxim Khitrov at Oct 5, 2012 at 11:06 am

    On Thu, Oct 4, 2012 at 11:05 PM, Monnand wrote:
    On 10/02/2012 02:30 PM, Maxim Khitrov wrote:
    <snip>
    The data pipeline for an IMAP connection looks like this:


    bufio <-> [flate] <-> [tls] <-> net.Conn

    The two middle stages (compression and encryption) are optional.
    Through some testing I found that the flate package doesn't play well
    with temporary errors. After a single timeout, the connection became
    unusable. As a result, I needed a timeout mechanism that bypassed the
    entire receive pipeline.

    I use SetDeadline when initiating and closing the connection. The
    goroutine solution seems safer while the connection is active. It lets
    me treat any error from net.Conn as fatal.

    - Max
    Let me see if I get it correct: you said "After a single timeout, the
    connection became unusable." So that means, if a Read/Write() returns a
    timeout error because of setting SetDeadline, I can no longer use the whole
    connection, am I correct?

    This seems not reasonable. I mean the behavior of the library should not be
    like this. Is this a feature or a bug of the standard library? Can you
    reproduce this scenario on every timeout?
    That's just the way the flate package works. Here's a short demo that
    simulates a timeout error on the second Read call:

    http://play.golang.org/p/cyyZlARsgl

    Decompressor remembers the last error returned from the underlying
    Reader and continues returning that error for all subsequent Read
    calls:

    https://code.google.com/p/go/source/browse/src/pkg/compress/flate/inflate.go#282

    I don't know if this is a bug or not. It's possible that an unexpected
    error could corrupt the internal decompressor state, making it
    impossible to continue reading.

    There is also another minor inconvenience in that if the other side
    doesn't properly terminate the compressed stream, the decompressor
    converts io.EOF to io.ErrUnexpectedEOF. You can simulate this by
    commenting out the Close() call. This isn't always the right thing to
    do, especially with network communications. Google's IMAP servers, for
    example, just close the connection without terminating the
    compression.

    - Max

    --

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedOct 2, '12 at 4:38p
activeOct 5, '12 at 11:06a
posts6
users2
websitegolang.org

2 users in discussion

Monnand: 3 posts Maxim Khitrov: 3 posts

People

Translate

site design / logo © 2022 Grokbase