FAQ
Hi everyone,

I'm implementing an HTTP service in Go where I receive a potentially
long-running file upload from a user. I want to time out if I do not
receive data from the user for more than N seconds, but as long as I'm
receiving data I want the request to continue indefinitely. What's the best
way to do this? Currently I'm using hijacker as in the example at
http://golang.org/pkg/net/http/#Hijacker. I hijack the connection at the
top of my Handler and I call
conn.SetReadDeadline(time.Now().Add(readIdleTimeout)) repeatedly while I
loop over getting data out of the bufrw.

There are two problems I see with this, one minor and the other less minor.
The minor one is that I have to inject myself into the buffer flow by
manually pulling data out of bufrw and putting it into a new buffer
(instead of just giving the original bufrw to the code needing the uploaded
data), which is slightly less efficient. The less minor one is that (as I
understand it) with hijacking I lose the ability to reuse connections, and
if I use SSL then reconstructing the request for every upload can get very
expensive.

Is there a better way to do this? If it's helpful I can put together an
example but I was hoping this would be enough info. The most ideal solution
in my mind would be to implement a ReadWriteTimeout in http.Server which
operates not on an absolute basis but instead on time-since-last-data-seen
basis.

Thanks!

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Search Discussions

  • James Bardin at May 3, 2014 at 1:07 am
    Here's an example I put together
    before https://gist.github.com/jbardin/9663312

    Just pass a custom listener to the http.Server.

    On Friday, May 2, 2014 5:23:53 PM UTC-4, s...@box.com wrote:

    Hi everyone,

    I'm implementing an HTTP service in Go where I receive a potentially
    long-running file upload from a user. I want to time out if I do not
    receive data from the user for more than N seconds, but as long as I'm
    receiving data I want the request to continue indefinitely. What's the best
    way to do this? Currently I'm using hijacker as in the example at
    http://golang.org/pkg/net/http/#Hijacker. I hijack the connection at the
    top of my Handler and I call
    conn.SetReadDeadline(time.Now().Add(readIdleTimeout)) repeatedly while I
    loop over getting data out of the bufrw.

    There are two problems I see with this, one minor and the other less
    minor. The minor one is that I have to inject myself into the buffer flow
    by manually pulling data out of bufrw and putting it into a new buffer
    (instead of just giving the original bufrw to the code needing the uploaded
    data), which is slightly less efficient. The less minor one is that (as I
    understand it) with hijacking I lose the ability to reuse connections, and
    if I use SSL then reconstructing the request for every upload can get very
    expensive.

    Is there a better way to do this? If it's helpful I can put together an
    example but I was hoping this would be enough info. The most ideal solution
    in my mind would be to implement a ReadWriteTimeout in http.Server which
    operates not on an absolute basis but instead on time-since-last-data-seen
    basis.

    Thanks!
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Sam Ghods at May 9, 2014 at 5:55 am
    Wow, this is *exactly* what I was looking for. I had a hard time
    discovering it because I was busy looking at the s := &http.Server{...}
    example on http://golang.org/pkg/net/http/ which makes it seem like the
    only custom thing you can set is myHandler. Thank you so much!

    On Fri, May 2, 2014 at 6:06 PM, James Bardin wrote:

    Here's an example I put together before
    https://gist.github.com/jbardin/9663312

    Just pass a custom listener to the http.Server.

    On Friday, May 2, 2014 5:23:53 PM UTC-4, s...@box.com wrote:

    Hi everyone,

    I'm implementing an HTTP service in Go where I receive a potentially
    long-running file upload from a user. I want to time out if I do not
    receive data from the user for more than N seconds, but as long as I'm
    receiving data I want the request to continue indefinitely. What's the best
    way to do this? Currently I'm using hijacker as in the example at
    http://golang.org/pkg/net/http/#Hijacker. I hijack the connection at the
    top of my Handler and I call conn.SetReadDeadline(time.Now().Add(readIdleTimeout))
    repeatedly while I loop over getting data out of the bufrw.

    There are two problems I see with this, one minor and the other less
    minor. The minor one is that I have to inject myself into the buffer flow
    by manually pulling data out of bufrw and putting it into a new buffer
    (instead of just giving the original bufrw to the code needing the uploaded
    data), which is slightly less efficient. The less minor one is that (as I
    understand it) with hijacking I lose the ability to reuse connections, and
    if I use SSL then reconstructing the request for every upload can get very
    expensive.

    Is there a better way to do this? If it's helpful I can put together an
    example but I was hoping this would be enough info. The most ideal solution
    in my mind would be to implement a ReadWriteTimeout in http.Server which
    operates not on an absolute basis but instead on time-since-last-data-seen
    basis.

    Thanks!

    --
    Sam

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Karan Misra at May 9, 2014 at 7:26 am
    Thanks for that :)
    On Saturday, May 3, 2014 6:36:57 AM UTC+5:30, James Bardin wrote:

    Here's an example I put together before
    https://gist.github.com/jbardin/9663312

    Just pass a custom listener to the http.Server.

    On Friday, May 2, 2014 5:23:53 PM UTC-4, s...@box.com wrote:

    Hi everyone,

    I'm implementing an HTTP service in Go where I receive a potentially
    long-running file upload from a user. I want to time out if I do not
    receive data from the user for more than N seconds, but as long as I'm
    receiving data I want the request to continue indefinitely. What's the best
    way to do this? Currently I'm using hijacker as in the example at
    http://golang.org/pkg/net/http/#Hijacker. I hijack the connection at the
    top of my Handler and I call
    conn.SetReadDeadline(time.Now().Add(readIdleTimeout)) repeatedly while I
    loop over getting data out of the bufrw.

    There are two problems I see with this, one minor and the other less
    minor. The minor one is that I have to inject myself into the buffer flow
    by manually pulling data out of bufrw and putting it into a new buffer
    (instead of just giving the original bufrw to the code needing the uploaded
    data), which is slightly less efficient. The less minor one is that (as I
    understand it) with hijacking I lose the ability to reuse connections, and
    if I use SSL then reconstructing the request for every upload can get very
    expensive.

    Is there a better way to do this? If it's helpful I can put together an
    example but I was hoping this would be enough info. The most ideal solution
    in my mind would be to implement a ReadWriteTimeout in http.Server which
    operates not on an absolute basis but instead on time-since-last-data-seen
    basis.

    Thanks!
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedMay 2, '14 at 9:23p
activeMay 9, '14 at 7:26a
posts4
users3
websitegolang.org

People

Translate

site design / logo © 2022 Grokbase