FAQ
http.Server has a parameter ReadTimeout time <http://golang.org/pkg/time/>.
Duration <http://golang.org/pkg/time/#Duration> // maximum duration before
timing out read of the request

This will timeout even when the connection is busy. What's more needed is a
timeout when the connection is idle. I wonder why there is no such way to
configure http.Server.

In my case, the client can be uploading a large file to the system, which
can take fairly long. Having a large ReadTimeout will work for large files,
but not so efficient for other smaller uploads.

Chris
http://weed-fs.googlecode.com

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Search Discussions

  • James Bardin at Mar 20, 2014 at 1:10 pm
    The pattern I've used is to set read and write deadlines one the underlying
    connection.

    You can do this using a custom net.Listener you hand off to the
    http.Server. It creates a connection which sets a deadline for every
    individual read and write, so you don't need to worry about the overall
    request time.

    https://gist.github.com/jbardin/9663312



    On Thursday, March 20, 2014 3:40:16 AM UTC-4, ChrisLu wrote:

    http.Server has a parameter ReadTimeout time <http://golang.org/pkg/time/>
    .Duration <http://golang.org/pkg/time/#Duration> // maximum duration
    before timing out read of the request

    This will timeout even when the connection is busy. What's more needed is
    a timeout when the connection is idle. I wonder why there is no such way to
    configure http.Server.

    In my case, the client can be uploading a large file to the system, which
    can take fairly long. Having a large ReadTimeout will work for large files,
    but not so efficient for other smaller uploads.

    Chris
    http://weed-fs.googlecode.com
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • ChrisLu at Mar 20, 2014 at 5:07 pm
    This is really great! I think every http.Server should use this idle
    timeout connection.

    Chris
    On Thursday, March 20, 2014 6:09:49 AM UTC-7, James Bardin wrote:

    The pattern I've used is to set read and write deadlines one the
    underlying connection.

    You can do this using a custom net.Listener you hand off to the
    http.Server. It creates a connection which sets a deadline for every
    individual read and write, so you don't need to worry about the overall
    request time.

    https://gist.github.com/jbardin/9663312



    On Thursday, March 20, 2014 3:40:16 AM UTC-4, ChrisLu wrote:

    http.Server has a parameter ReadTimeout time<http://golang.org/pkg/time/>
    .Duration <http://golang.org/pkg/time/#Duration> // maximum duration
    before timing out read of the request

    This will timeout even when the connection is busy. What's more needed is
    a timeout when the connection is idle. I wonder why there is no such way to
    configure http.Server.

    In my case, the client can be uploading a large file to the system, which
    can take fairly long. Having a large ReadTimeout will work for large files,
    but not so efficient for other smaller uploads.

    Chris
    http://weed-fs.googlecode.com
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Benjamin Measures at Mar 20, 2014 at 6:08 pm

    On Thursday, 20 March 2014 17:07:27 UTC, ChrisLu wrote:

    This is really great! I think every http.Server should use this idle
    timeout connection.
    This would make it susceptable to attacks like slowris (though cheap
    goroutines do help mitigate somewhat).

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Benjamin Measures at Mar 20, 2014 at 6:09 pm

    On Thursday, 20 March 2014 18:08:04 UTC, Benjamin Measures wrote:
    On Thursday, 20 March 2014 17:07:27 UTC, ChrisLu wrote:

    This is really great! I think every http.Server should use this idle
    timeout connection.
    This would make it susceptable to attacks like slowris (though cheap
    goroutines do help mitigate somewhat).
    *Slowloris

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • James Bardin at Mar 20, 2014 at 6:31 pm

    On Thu, Mar 20, 2014 at 2:09 PM, Benjamin Measures wrote:

    This would make it susceptable to attacks like slowris (though cheap
    goroutines do help mitigate somewhat).
    *Slowloris
    An overall timeout is still a good idea in many cases.
    Accepting uploads from mobile clients is often not much different than a
    slowloris attack anyway, so have to try and accept that.

    It's still useful in combination with a very long ReadTimeout, because this
    lets you more quickly free up extra resources that legit clients may be
    holding if they lose their connection (e.g. a lock, a db setting for "in
    progress", an open file for upload, etc)

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Benjamin Measures at Mar 20, 2014 at 9:33 pm

    On Thursday, 20 March 2014 18:31:05 UTC, James Bardin wrote:
    On Thu, Mar 20, 2014 at 2:09 PM, Benjamin Measures <saint....@gmail.com<javascript:>
    wrote:
    *Slowloris
    An overall timeout is still a good idea in many cases.
    Accepting uploads from mobile clients is often not much different than a
    slowloris attack anyway, so have to try and accept that.
    To some extent. I doubt real people wait >2m for a page to show though.

    It's still useful in combination with a very long ReadTimeout, because this
    lets you more quickly free up extra resources that legit clients may be
    holding if they lose their connection (e.g. a lock, a db setting for "in
    progress", an open file for upload, etc)
    Certainly. Care must be taken though to ensure that your extending
    net.Conn.SetReadDeadline doesn't interfere with http.Server.ReadTimeout,
    which itself is implemented by net.Conn.SetReadDeadline.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Gyepi SAM at Mar 21, 2014 at 6:31 am

    On Thu, Mar 20, 2014 at 02:33:51PM -0700, Benjamin Measures wrote:
    To some extent. I doubt real people wait >2m for a page to show though.
    I can assure you, real people do wait. Just not real people who expect their
    connections to be nearly instantaneous or at least fast.

    Internet speeds can be quite slow in many parts of the world.

    I'm currently in South Africa and have learned to temper my impatience
    (and change my browsing habits).

    -Gyepi

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • James Bardin at Mar 21, 2014 at 1:56 pm

    On Thursday, March 20, 2014 5:33:51 PM UTC-4, Benjamin Measures wrote:

    To some extent. I doubt real people wait >2m for a page to show though.
    Well, we are talking more about long requests here, like uploads. Not all
    http requests need to result in rendering a web page.

    It's still useful in combination with a very long ReadTimeout, because
    this lets you more quickly free up extra resources that legit clients may
    be holding if they lose their connection (e.g. a lock, a db setting for "in
    progress", an open file for upload, etc)
    Certainly. Care must be taken though to ensure that your extending
    net.Conn.SetReadDeadline doesn't interfere with http.Server.ReadTimeout,
    which itself is implemented by net.Conn.SetReadDeadline.

    Good point. I was thinking that http.Server used a timer to close the
    connection, but I may have had another part the http package in my head.
    Yes, it looks like this will override the ReadTimeout setting since I'm
    setting the deadline at the actual time of Read, so you'll probably need to
    set a timer to force the connection closed if it's a concern.

    maybe something like this if you want to fit it into the pattern I posted
    (warning, untested) http://play.golang.org/p/hKWv_83eGx


    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Ox at Aug 18, 2014 at 8:04 pm
    I've started using this <https://github.com/getlantern/idletiming> to add
    idle timeouts to connections and listeners.
    On Friday, March 21, 2014 8:55:57 AM UTC-5, James Bardin wrote:


    On Thursday, March 20, 2014 5:33:51 PM UTC-4, Benjamin Measures wrote:


    To some extent. I doubt real people wait >2m for a page to show though.
    Well, we are talking more about long requests here, like uploads. Not all
    http requests need to result in rendering a web page.

    It's still useful in combination with a very long ReadTimeout, because
    this lets you more quickly free up extra resources that legit clients may
    be holding if they lose their connection (e.g. a lock, a db setting for "in
    progress", an open file for upload, etc)
    Certainly. Care must be taken though to ensure that your extending
    net.Conn.SetReadDeadline doesn't interfere with http.Server.ReadTimeout,
    which itself is implemented by net.Conn.SetReadDeadline.

    Good point. I was thinking that http.Server used a timer to close the
    connection, but I may have had another part the http package in my head.
    Yes, it looks like this will override the ReadTimeout setting since I'm
    setting the deadline at the actual time of Read, so you'll probably need to
    set a timer to force the connection closed if it's a concern.

    maybe something like this if you want to fit it into the pattern I posted
    (warning, untested) http://play.golang.org/p/hKWv_83eGx
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • James Bardin at Aug 18, 2014 at 8:55 pm

    On Monday, August 18, 2014 4:04:25 PM UTC-4, o...@getlantern.org wrote:
    I've started using this <https://github.com/getlantern/idletiming> to add
    idle timeouts to connections and listeners.
    I wouldn't use that as a drop-in replacement without testing things first.

    It doesn't use the Read/Write deadlines for the timeout, it closes the
    connection out from underneath you which you may not be expecting -- you
    can't check for a net.Error.Timeout(). Also, because it doesn't use the
    deadlines directly, it's polling and will take up to 2*idleTimeout to
    actually close the connection.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Ox Cart at Aug 18, 2014 at 9:21 pm
    Thanks, you're right that the actual timeout may be up to twice the
    specified value. I've updated the comment accordingly.

    Regarding not setting read/write deadlines directly, that was intentional.
      From what I recall, those calls are actually somewhat expensive, so doing
    that on every read/write was inefficient.

    Cheers,
    Ox

    -----------------------------------------------------------------------------------------

    "I love people who harness themselves, an ox to a heavy cart,
    who pull like water buffalo, with massive patience,
    who strain in the mud and the muck to move things forward,
    who do what has to be done, again and again."

    - Marge Piercy


    On Mon, Aug 18, 2014 at 3:55 PM, James Bardin wrote:


    On Monday, August 18, 2014 4:04:25 PM UTC-4, o...@getlantern.org wrote:

    I've started using this <https://github.com/getlantern/idletiming> to
    add idle timeouts to connections and listeners.
    I wouldn't use that as a drop-in replacement without testing things first.

    It doesn't use the Read/Write deadlines for the timeout, it closes the
    connection out from underneath you which you may not be expecting -- you
    can't check for a net.Error.Timeout(). Also, because it doesn't use the
    deadlines directly, it's polling and will take up to 2*idleTimeout to
    actually close the connection.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • James Bardin at Aug 18, 2014 at 9:46 pm

    On Mon, Aug 18, 2014 at 5:21 PM, Ox Cart wrote:

    Thanks, you're right that the actual timeout may be up to twice the
    specified value. I've updated the comment accordingly.

    Regarding not setting read/write deadlines directly, that was intentional.
    From what I recall, those calls are actually somewhat expensive, so doing
    that on every read/write was inefficient.

    I don't think those calls should have that much impact. When doing this on
    a high-throughput server I made sure to use a *net.TCPConn directly,
    because the real hit was in resolving the method from the interface, but
    that seems to no longer be an issue in more recent go versions and I just
    use net.Conn throughout.

    At least on the systems I'm using, moving the deadline is so much faster
    than actual writes to the network, there's not any measurable impact.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Ox Cart at Aug 18, 2014 at 9:49 pm
    Nice! I seem to remember profiling that approach a while back and seeing a
    fair amount of overhead associated with it, but unfortunately I don't
    remember specifics. When I have some time, I'll try it out again.

    Cheers,
    Ox

    -----------------------------------------------------------------------------------------

    "I love people who harness themselves, an ox to a heavy cart,
    who pull like water buffalo, with massive patience,
    who strain in the mud and the muck to move things forward,
    who do what has to be done, again and again."

    - Marge Piercy


    On Mon, Aug 18, 2014 at 4:40 PM, James Bardin wrote:



    On Mon, Aug 18, 2014 at 5:21 PM, Ox Cart wrote:

    Thanks, you're right that the actual timeout may be up to twice the
    specified value. I've updated the comment accordingly.

    Regarding not setting read/write deadlines directly, that was
    intentional. From what I recall, those calls are actually somewhat
    expensive, so doing that on every read/write was inefficient.

    I don't think those calls should have that much impact. When doing this on
    a high-throughput server I made sure to use a *net.TCPConn directly,
    because the real hit was in resolving the method from the interface, but
    that seems to no longer be an issue in more recent go versions and I just
    use net.Conn throughout.

    At least on the systems I'm using, moving the deadline is so much faster
    than actual writes to the network, there's not any measurable impact.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedMar 20, '14 at 7:40a
activeAug 18, '14 at 9:49p
posts14
users5
websitegolang.org

People

Translate

site design / logo © 2021 Grokbase