FAQ
Should I completely read the body of response (my be copy into
ioutil.Discard), even if I'm not interested in it, for http.Client to
re-use connections?

--
Chandra Sekar.S

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Search Discussions

  • Dave Cheney at Dec 10, 2014 at 1:50 pm
    Yes, and if you don't resp.Body.Close() will do it for you.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • James Bardin at Dec 10, 2014 at 2:27 pm

    On Wednesday, December 10, 2014 8:49:59 AM UTC-5, Dave Cheney wrote:
    Yes, and if you don't resp.Body.Close() will do it for you.

    I'm fairly certain that's not true, otherwise closing an unneeded response
    could read from the connection indefinitely.

    I think it does get read implicitly if it fits entirely in the read buffer,
    but there's no guarantee. Reading it all will ensure that the transport
    attempts to reuse the connection.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Chandru at Dec 10, 2014 at 2:35 pm
    After some digging into stdlib code,
    http://golang.org/src/pkg/net/http/transport.go?h=bodyEOFSignal#L1112 and
    http://golang.org/src/pkg/net/http/transport.go?h=bodyEOFSignal#L841
    suggest the connection will be closed if body is not read till end.

    Am I reading it wrong?

    --
    Chandra Sekar.S
    On Wed, Dec 10, 2014 at 7:19 PM, Dave Cheney wrote:

    Yes, and if you don't resp.Body.Close() will do it for you.

    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • James Bardin at Dec 10, 2014 at 2:39 pm

    On Wed, Dec 10, 2014 at 9:34 AM, Chandru wrote:

    Am I reading it wrong?
    No, you are correct. Some of the body may be read into a buffer, but
    there's no guarantee how much.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Dave Cheney at Dec 10, 2014 at 8:16 pm
    I am sorry. This logic has changed in the past a few times and I am out of date.

    I think the current logic is

    io.Copy(ioutil.Discard, resp.Body); resp.Body.Close() // connection
    will be reused
    resp.Body.Close() // remaining body will be discarded, and the
    connection will _not_ be reused.
    On Thu, Dec 11, 2014 at 1:39 AM, James Bardin wrote:
    On Wed, Dec 10, 2014 at 9:34 AM, Chandru wrote:

    Am I reading it wrong?

    No, you are correct. Some of the body may be read into a buffer, but there's
    no guarantee how much.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Cbroglie at Aug 24, 2015 at 4:49 pm
    Is it considered a best practice to always add the following after
    http.DefaultClient.Do returns a non-error response?

    defer res.Body.Close()
    defer io.Copy(ioutil.Discard, res.Body)

    That way subsequent code in the function can read the body however it sees
    fit, but if it doesn't read it to completion for some reason, we cleanup
    and make sure the connection can be re-used. And it's a no-op if the body
    was already read fully.
    On Wednesday, December 10, 2014 at 12:17:26 PM UTC-8, Dave Cheney wrote:

    I am sorry. This logic has changed in the past a few times and I am out of
    date.

    I think the current logic is

    io.Copy(ioutil.Discard, resp.Body); resp.Body.Close() // connection
    will be reused
    resp.Body.Close() // remaining body will be discarded, and the
    connection will _not_ be reused.

    On Thu, Dec 11, 2014 at 1:39 AM, James Bardin <j.ba...@gmail.com
    <javascript:>> wrote:
    On Wed, Dec 10, 2014 at 9:34 AM, Chandru <chand...@gmail.com
    <javascript:>> wrote:
    Am I reading it wrong?

    No, you are correct. Some of the body may be read into a buffer, but there's
    no guarantee how much.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • James Bardin at Aug 24, 2015 at 5:04 pm

    On Mon, Aug 24, 2015 at 12:48 PM, wrote:

    Is it considered a best practice to always add the following after
    http.DefaultClient.Do returns a non-error response?

    defer res.Body.Close()
    defer io.Copy(ioutil.Discard, res.Body)

    That way subsequent code in the function can read the body however it sees
    fit, but if it doesn't read it to completion for some reason, we cleanup
    and make sure the connection can be re-used. And it's a no-op if the body
    was already read fully.
    defer'ing the Body.Close is definitely a best practice.

    You can add the io.Copy call if it suits your code, though I can't think of
    many cases where I didn't know if I would read the Body or not. While not
    exactly a noop, I don't think it ever really hurts to add it when you might
    need it, since ioutil.Discard gets its copy buffer from a sync.Pool.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Cbroglie at Aug 24, 2015 at 5:21 pm
    Thanks. There are 3 common cases I've encountered where the io.Copy is
    needed:
    - Reading the body using json.Decoder - it can stop without reading the
    entire body contents
    - Early return without reading the body b/c of a non-200 response
    - Client code doing a POST where it checks the http response code without
    reading the body

    Deferring the io.Copy at the same spot as the Close lets me ensure the body
    is always read without having to manually review each error case
    On Monday, August 24, 2015 at 10:04:53 AM UTC-7, James Bardin wrote:

    On Mon, Aug 24, 2015 at 12:48 PM, <cbro...@gmail.com <javascript:>> wrote:

    Is it considered a best practice to always add the following after
    http.DefaultClient.Do returns a non-error response?

    defer res.Body.Close()
    defer io.Copy(ioutil.Discard, res.Body)

    That way subsequent code in the function can read the body however it
    sees fit, but if it doesn't read it to completion for some reason, we
    cleanup and make sure the connection can be re-used. And it's a no-op if
    the body was already read fully.
    defer'ing the Body.Close is definitely a best practice.

    You can add the io.Copy call if it suits your code, though I can't think
    of many cases where I didn't know if I would read the Body or not. While
    not exactly a noop, I don't think it ever really hurts to add it when you
    might need it, since ioutil.Discard gets its copy buffer from a sync.Pool.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • James Bardin at Aug 24, 2015 at 5:49 pm

    On Mon, Aug 24, 2015 at 1:21 PM, wrote:

    Thanks. There are 3 common cases I've encountered where the io.Copy is
    needed:
    - Reading the body using json.Decoder - it can stop without reading the
    entire body contents
    - Early return without reading the body b/c of a non-200 response
    - Client code doing a POST where it checks the http response code without
    reading the body

    Deferring the io.Copy at the same spot as the Close lets me ensure the
    body is always read without having to manually review each error case
    All good points.

    A couple counterpoints to using this universally:

    - You may not want to wait for the full response, and discarding the
    connection is better or faster for various reasons. You could dispatch it
    in another goroutine, but then you need to move the Close to that
    goroutine, it adds a little more overhead, and wastes bandwidth for large
    responses.

    - You need to watch out for unexpectedly large responses, though you can
    mitigate this somewhat by wrapping the resp.Body in an io.LimitedReader.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Cbroglie at Aug 24, 2015 at 6:41 pm
    Good point about large responses
    On Monday, August 24, 2015 at 10:49:36 AM UTC-7, James Bardin wrote:

    On Mon, Aug 24, 2015 at 1:21 PM, <cbro...@gmail.com <javascript:>> wrote:

    Thanks. There are 3 common cases I've encountered where the io.Copy is
    needed:
    - Reading the body using json.Decoder - it can stop without reading the
    entire body contents
    - Early return without reading the body b/c of a non-200 response
    - Client code doing a POST where it checks the http response code without
    reading the body

    Deferring the io.Copy at the same spot as the Close lets me ensure the
    body is always read without having to manually review each error case
    All good points.

    A couple counterpoints to using this universally:

    - You may not want to wait for the full response, and discarding the
    connection is better or faster for various reasons. You could dispatch it
    in another goroutine, but then you need to move the Close to that
    goroutine, it adds a little more overhead, and wastes bandwidth for large
    responses.

    - You need to watch out for unexpectedly large responses, though you can
    mitigate this somewhat by wrapping the resp.Body in an io.LimitedReader.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedDec 10, '14 at 1:46p
activeAug 24, '15 at 6:41p
posts11
users4
websitegolang.org

People

Translate

site design / logo © 2021 Grokbase